![\"\"](\"https:\/\/abovethelaw.com\/wp-content\/uploads\/sites\/4\/2017\/12\/Ivy-IMG_3968-copy-300x300.jpg\")
Our approach to technology and process improvement initiatives is often fueled by hope, driven by shame, and made urgent by fearful hype.<\/p>\nOur approach to technology and process improvement initiatives is often fueled by hope, driven by shame, and made urgent by fearful hype.<\/p>\n
Missing from that list?<\/p>\n
Data.<\/p>\n
This does little to set us up for success and leaves change efforts subject to the whims of whomever is loudest or most powerful.\u00a0 If we want to see change in law firms, we must prove that tech initiatives deliver results.\u00a0 And if our change efforts are failing, then we must correct course or cut our losses.<\/p>\n
We can\u2019t do either without data.<\/p>\n
You Can\u2019t Manage What You Don\u2019t Measure<\/strong><\/p>\n We introduce technology and process improvement initiatives, but rarely track their progress. We don\u2019t know whether benefits outweigh costs\u2014or the costs outweigh benefits.<\/p>\n As a result, we:<\/p>\n \u2022 can\u2019t justify continuing current initiatives or launching new ones;<\/p>\n \u2022 don\u2019t have the data to encourage people to continue their journey;<\/p>\n \u2022 don\u2019t know how to improve what we are doing or know what needs improvement;<\/p>\n \u2022 can\u2019t make a well-informed decision to correct course or cut our losses.<\/p>\n We must measure and track what we\u2019re doing.<\/p>\n The good news is this concept is not new.\u00a0 Evaluation is its own discipline and it has been embraced successfully in industries worldwide. It\u2019s time for the legal industry to apply data-driven evaluation to technology.<\/p>\n A Theory of Change <\/strong><\/p>\n To understand a technology intervention and evaluate implementation, consider what the intervention requires and what it\u2019s designed to produce. The inputs, activities, outputs, outcomes, and impacts that make a theory of change will vary with the circumstances. Within legal tech, a theory of change can usually be broken down along these lines:<\/p>\n Figure 1: Theory of Legal Tech Change<\/strong><\/p>\n Looking at the diagram, if we exclude the category of impacts (which are, by their nature, difficult to measure) and activities (which don\u2019t require quantification), every other element of this theory can be counted and measured. Data and metrics for legal tech are feasible, and fairly simple to collect.<\/p>\n To ensure that technology and process improvement initiatives are valuable, we must measure and record our change efforts and their results. Using metrics that track change is essential. It will allow you to accurately assess the situation, fix problems, and advance objectives. Metrics also serve as a compass throughout the change management process, which help to evaluate and adjust course.<\/p>\n Let\u2019s apply these five elements of change to a legal technology implementation or intervention.<\/p>\n Objectives <\/strong><\/p>\n Tracking the wrong data or creating an overly burdensome data collection process will thwart your efforts.\u00a0 So start by establishing and clarifying your business goals for this change effort. Then determine what conditions must be met to make it possible to meet your business goals. These are critical success factors (\u201cCSFs\u201d). Then implement and track your initiative. It should line up with your CSFs and your KPIs (defined below). Each element of your plan should have a change management outcome-focused metric. Adopt activity-based metrics, such as counting actions taken, training courses completed, time spent, etc. And also adopt outcome-based metrics to assess whether you are meeting your business goals. For each metric, define what success looks like at each stage of change. Assign someone to own the data collection for accountability.<\/p>\n Inputs <\/strong><\/p>\n Inputs include everything that goes into selecting, implementing, and maintaining a new technology or process. It includes search, scoping, bid evaluation, and selection; piloting; acquisition cost; training and support. The time and financial cost of each input must ultimately be compared with results. Remember to include the cost of evaluating and monitoring in your inputs. Assume between 1% and 5% of the total project budget for monitoring.<\/p>\n Outputs <\/strong><\/p>\n Outputs are direct metrics that can be clearly attributed to activities. For technology initiatives, you\u2019ll want to analyze usage patterns. How much is the new technology being used? Are users embracing the technology or begrudgingly using it? Is there a trend showing that adoption is growing? Metrics might include individual log-ons, calls to tech support increasing or decreasing, compliance with new processes, and incorporation into workflow. You might also want to conduct user surveys to determine whether users are getting frustrated, confused or lost; whether users are using workarounds that introduce error; or whether users are improperly delegating work to others. These metrics help to assess whether the technology initiative is working as planned.<\/p>\n Outcomes <\/strong><\/p>\n From there, you can develop your key performance indicators (\u201cKPIs\u201d).\u00a0 KPIs must be important, predictive, and measure achievement or success. They can relate to financial outcomes, client satisfaction, process improvement, or skills building, empowerment, or investment of individuals. Select no more than five KPIs. Your selection should include a combination of outcome metrics that give insight to the end result of a process and process metrics that give feedback on the performance elements of the process. Overall, KPIs should reflect progress toward a specified goal and align business activities and individual actions with strategic objectives. For law firms, one of the main KPIs is the realization rate. Realization rates reflect the real value of work to the firm and consider efficiency, experience, technology, staffing, and client satisfaction. However, if realization rates are your only KPI, you risk attributing too much to that one metric (and in the real world, metrics may change for many reasons), so always have multiple KPIs.<\/p>\n Monitoring, Evaluation, and Action <\/strong><\/p>\n Monitoring is making sure you get the data.\u00a0 But that\u2019s not enough. Using your data to clearly evaluate whether your intervention is working is key. Evaluation measures costs and benefits. If your measurement of outputs shows that an initiative is failing, maybe more training on the technology is required. Or maybe the people in your organization don\u2019t understand the benefit of the change. Uncovering the reason for the failure is key to figuring out how to fix it. With data, you can find the problem early, correct course, and make the change successful.<\/p>\n Celebrate Small Wins<\/strong><\/p>\n On your path to transforming your business, use metrics to track and celebrate small wins. Measurement can motivate people and give guidance toward goal achievement. Small wins give feedback and justification for the change effort and validate decisions and strategies. Small wins also build faith in the change effort and can even draw in people who hadn\u2019t yet embraced the initiative.<\/p>\n Cut Your Losses <\/strong><\/p>\n As much as we are hopeful, sometimes the best thing to do with a failing tech initiative is to end it. If four people are using an expensive piece of software bought for the entire firm, don\u2019t continue to pay for the entire firm to use it. Sometimes IT projects don\u2019t work out. If that\u2019s your firm, don\u2019t regret that\u2013at least your firm is taking the steps in the right direction and trying. But equally, don\u2019t continue to pour money into it. How often has the ego of a partner or IT guru led to big bills for a technology that no one wants or uses? Hanging on to a poor fit and a bad decision will hamper future efforts.<\/p>\n Find the Lesson <\/strong><\/p>\n When you\u2019ve successfully implemented technology and reaped the rewards, don\u2019t forget to evaluate your success so it can be repeated. Look for the lessons. Did benefits outweigh costs because cost was low or because benefits were high? What other kinds of technology intervention are worth considering for your firm as a result?<\/p>\n Conclusion <\/strong><\/p>\n Tracking the success of technology and process improvement initiatives is key to knowing whether they work and planning for the future. Metrics that show success can also justify the costs of ongoing technology investments and training programs, and support efforts to launch new initiatives. Capturing small wins can encourage individuals to continue their learning journey despite setbacks. And data can show whether technology initiatives are actually meeting business objectives. Collecting and evaluating data will help you to manage, measure, and motivate.<\/p>\n Ivy B. Grey is an accomplished lawyer and writer. She is the author of American Legal Style for PerfectIt, which is a proofreading and editing add-in for lawyers. Ivy is also a Senior Attorney at Griffin Hamersky LLP. She focuses her practice on bankruptcy, which includes distressed transactions and some litigation. She’s been named as a Rising Star in the New York Metro Area for four consecutive years, and her significant representations include In re AMR Corp. (American Airlines), In re Dewey & LeBoeuf LLP, In re Eastman Kodak Company, In re Filmed Entertainment Inc. (Columbia House), and In re Nortel Networks Inc. Ivy received her J.D. from the University of Houston Law Center where she was Chief Notes & Comments Editor of the Houston Business & Tax Law Journal. Prior to becoming a lawyer, Ivy spent about 10 years working in public relations and advertising. You can follow Ivy on twitter at @IvyBGrey or connect with her on LinkedIn at www.linkedin.com\/in\/ivybgrey\/.<\/strong><\/em><\/p>\n","protected":false},"excerpt":{"rendered":" Tracking the success of technology and process improvement initiatives is key to knowing whether they work and planning for the future. <\/p>\n","protected":false},"author":71800,"featured_media":1799,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2366,2508,2563,2974,2191,2395,11],"tags":[7],"class_list":["post-3258","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data","category-innovation","category-legal-design","category-legal-technology","category-new-ideas","category-startup","category-technology","tag-technology"],"acf":[],"yoast_head":"\n![\"\"](\"https:\/\/abovethelaw.com\/wp-content\/uploads\/sites\/4\/2018\/05\/Screen-Shot-2018-05-09-at-10.58.13-AM-300x117.png\")
<\/p>\n
\n