Authors

  • Nick Hart, Ph.D., President, Data Foundation

  • Adita Karkera, Chief Data Officer, Deloitte Consulting LLP Government & Public Services

  • Valerie Logan, CEO and Founder, The Data Lodge

Executive Summary

Advances in the access, collection, management, analysis, and use of data across public sector organizations substantially contributed to steady improvements in services, efficiency of operations, and effectiveness of government programs. The experience of citizens, beneficiaries, managers, and data experts is also evolving as data becomes pervasive and more seamlessly integrated within decision-making processes. In order for agencies to effectively engage in the ever-changing data landscape, organizational data literacy capacity and program models can help ensure individuals across the workforce can read, write, and communicate with data in the context of their role.

Data and analytics are no longer “just” for specialists, such as data engineers and data scientists; rather, data literacy is now increasingly recognized as a core workforce competency. Fortunately, in the United States several pioneers have emerged in strategically advancing data literacy programs and activities at the organizational level, providing benefits to individuals in the public sector workforce. Pioneering programs are those that recognize data literacy as more than training. They view data literacy as a holistic set of activities program to engage employees at all levels with data, develop employees with relevant skills, and enable scale of data literacy by augmenting employees’ skills with guided learning support and resources.

Agencies should begin by crafting the case for change. As is common with any emerging field, varying definitions and interpretations of “data literacy” are prevalent, which can affect program design. Being explicit in what problems are being solved for, as well as the needs and drivers to be addressed with a data literacy program or capacity, are vital to mitigate false starts. Looking across the ten pioneers discussed in this report, key lessons emerged that are relevant for government agencies as they design data literacy capacity and programs:

  • Engage Senior Leaders with Clear Roles and Expectations. Executive leaders set the tone for data literacy and data use in an organization, so ensuring the leadership is clear about shared and individual responsibilities also enables chief data officers, human capital officials, and others to sponsor and implement the data literacy program effectively.

  • Clarify Target Competencies and Personas for Actionable Gap Analysis. Thoughtful approaches to building data literacy programs include the use of data in assessing existing competencies, personas, capacity, and skills across the workforce, then identifying strategic areas for investment and improvement based on organizational priorities.

  • Cultivate a Common, Shared Language. Talking about data in a common, relatable way across an organization can drive traction in the application of data skills and expertise for lasting impact.

  • Improve Data Accessibility. Even with a confident and capable workforce, if data access is unintentionally restrictive and data cannot be readily used in practice, data literacy programs will be limited in their ability to foster a data-informed culture across the agency.

  • Align Data Governance and Data Literacy. As agencies are maturing data governance policies and practices, data literacy should be adjusted in tandem and vice versa. Key data governance work products including a data catalog, glossary, and
    data dictionary become critical foundations and dependencies for a successful data literacy initiative.

  • Encourage the Use of Data in Decision-Making. As organizations build data literacy programs and advance data governance processes, senior leaders will increasingly be positioned to highlight the explicit impact and value of data in decision-making at all levels of the organization.

For organizations seeking to advance data literacy programs and build capacity, the lessons also suggest a series of actionable recommendations, including:

  • Sponsorship. Agency heads should designate a Chief Data Officer or other official to sponsor the data literacy program and allocate sufficient resources for the initiative, including to staff a program lead position, provide a program budget, support public- private partnerships, and to continually analyze staff needs as the program matures.

  • Transparency. In support of the case for how data literacy supports agency goals, government executives should be more transparent and illustrative in how they use data with agency staff, and support identifying the data gaps through a learning agenda.

  • Incentives. As agency data literacy programs mature, agencies should intentionally reinforce the cultural values of data- and evidence-informed decisions with incentives for the use of data in grants, regulations, and policy guidance.

    The pioneers discussed in this report offer early lessons as other organizations also seek to improve capabilities for using data. Recognizing that program development is a learning process also means that no agency or organization should aspire for a perfect program at the outset; every program will change over time. What is essential is that every organization begins to build its capacity for using data and evidence—and that all starts with data literacy.

Introduction

Advances in the access, collection, management, analysis, and use of data across public sector organizations substantially contributed to steady improvements in services, efficiency of operations, and effectiveness of government programs. In virtually every level of government, constituents expect their government to responsibly apply insights from data to achieve goals and fulfill anticipated outcomes. Yet the practical application of data analysis to solve problems requires more than expectations—it involves a skilled and collaborative workforce that is both willing and able to effectively manage and use valid, reliable data that is fit for purpose. Responsible applications of data also require a workforce to know when data cannot be used for a particular purpose, understanding the caveats and limitations, and even the knowledge gaps that might exist from a data resource.

The COVID-19 pandemic placed a spotlight not only on where basic data infrastructure around the world can or should be improved, but also on areas where additional support, training, and focus might exist to enable public sector organizations to quickly understand how to identify data needs, how to formulate and act upon strategies to address those needs, and then how to ensure accurate, valid analysis and interpretation occurs. At the outset of the pandemic in the United States, reporting on positive COVID-19 cases and mortality experienced data quality limits, time lags, and misreporting that affected community, state, and national responses. Within months, agencies and communities in the U.S. began making improvements to that infrastructure, producing a national system that included predictive modeling for caseloads and fatalities, including simple reports provided to governors and key decision-makers. Media developed accessible strategies for the public to interpret similar information. Today, many in the U.S. and around the world recognize the value of reviewing caseloads with a 7-day average and the value of normalizing reports to control for population in a particular geographic area. These are examples where many in our society benefited from increased awareness and education about how to interpret information that matters to the lives of those the government serves. This was a teaching moment on the need for and critical role of data literacy that spanned our professional and personal lives.

In practice, everyone can develop some level of awareness for how data shapes and affects their activities, behaviors, and lives. Public sector organizations have a unique responsibility on behalf of citizens and constituents they serve to support informed decisions equitably across the population. This often means policy and regulatory actions involve transparency about decisions, accessibility to resources used to make decisions, and even participatory processes to engage on the meaning of available information in decision-making. For those making decisions with this expectation in public sector organizations, awareness about the role of data is definitely important, though only part of the responsibility; a much more extensive appreciation of data literacy is needed across the workforce of public sector organizations.

The challenges facing the public sector workforce’s ability to use data at every level of government around the world—whether local, regional, or national—are very real. Some of these challenges are institutional, regulatory, or legal. Others are based on capacity constraints or skills gaps in the workforce. Recognizing or acknowledging the many challenges is a relatively simple task. The more complicated endeavor is identifying the set of solutions and approaches for effectively building a public sector workforce and organizations that are responsible stewards and consumers of data. The solution orientation is precisely what this paper seeks to present in explaining how data literacy contributes to the success of public sector organizations and practical steps and best practices for improving data literacy.

We begin by explaining the meaning of data literacy and address some of the term’s myths and misconceptions. We describe why data literacy matters for public sector organizations and discuss the factors and enablers for success in data literate organizations. Specific examples of organizations in the U.S. that are pioneering early data literacy programs are presented, each serving as a potential model for how public sector organizations can begin to improve data literacy across their workforce. We also provide a realistic perspective on the challenges organizations face when attempting to foster data literacy that inform our suggestions for how public sector organizations can begin the process of prioritizing literacy at all levels of planning and operations.

Explaining Data Literacy: Developing a Shared Definition

While there are varying definitions of the term “data literacy,” in this paper we intend the concept to specifically describe an individual’s ability to read, write, and communicate with data in context.[1] Other definitions also exist. For example, Deloitte defines data literacy as the ability to read, work with, analyze and use data ethically to solve challenges, drive innovation and create value collaboratively. In particular, the CDO Council at Deloitte has been focused on creating additional rigor around data literacy to build awareness of the value of data-driven decision-making and to identify critical pathways for developing proficiency in data literacy in their government and commercial CDO workshops. In practice, literacy has real-world implications that affect how individuals ethically process and meaningfully use data.

Data literacy at an organizational level consists of individuals within the organization coordinating, collaborating, and engaging to use data across the organizational culture. While organizations in the public or private sector may recruit data literate employees, embedding the concept of data literacy into organizational culture so that it is pervasive across all units and employees in an organization requires more intentional effort. “Data literate organizations” are those that operate with data embedded in core decision- making processes, where data is valued by organizational leaders and line employees alike, and where intentional and effective efforts to improve and sustain data literacy are prioritized with activities that build capacity, ideally with a comprehensive data literacy program.

A “data literacy program” is a component of organizational capacity building. A data literacy program is an intentional and sustained commitment from organizational leaders to operationalize efforts that results in upskilling the workforce, improving
data understanding and use, and enabling workers in the organization to use relevant data analytics for key decision moments. As has been widely discussed and developed as a model by The Data Lodge, successful data literacy programs are built on three core types of activities: engagement, development, and enablement.[2]

Another way to productively conceptualize data literacy programs is through the activities that are offered, and how those activities lead to organizational change and resulting impacts. If data literacy programs include training, development, and availability of tools, then the outputs from those various activities should result in more informed analysis, or perhaps more analysis in general based on increased access to data, information, and evidence.

The reasonable expectation is that the availability of evidence changes behaviors because it can be used in decisions, including because of leadership support for data literacy programs. In some cases, this may result in an identification of where new data or analysis is still needed, because decision makers are uncomfortable with the quality or the level of granularity in a particular analysis. Finally, this leads to organizational outcomes that relate to the mission, for example, increased efficiency or effectiveness, and reduced improper payment or fraud cases.

What is clear is that the concept of data literacy is one that continues to rapidly evolve based on experiences and learning of individuals and organizations alike. Data literacy is relevant for organizations just as much as for the individuals within those organizations, and an organization cannot become data literate without emphasizing data literacy for their employees. This is precisely the value the data literacy program brings to bear, harmonizing and organizing the array of activities into a coherent and knowable process to support planning, decisions, and actions. If the activities are implemented, through a program or otherwise, the theory of change suggests clear and positive benefits to public sector organizations.

DATA LITERACY GENERIC THEORY OF CHANGE

 

“By adding data literacy, shared concepts and a common data language, we enrich our data & insights-driven culture, which gets embedded into our Firm’s learning fabric for ethics and digital journey acceleration.”

—CARL GERBER, Audit & Assurance Partner and Deloitte Chief Data Officer

Data Literacy Realizes the Strategic Value of Government Data

Public sector organizations frequently collect and manage data through the course of program operations. This information is then accessed and analyzed to support program decision-making and to improve policy outcomes, yet many governments globally at the national, regional, and local level struggle to effectively use data assets they already manage or have access to from additional sources. This also means that many public sector organizations are likely not realizing the optimal return on the often-substantial investments for collecting data that could be useful to government officials for program implementation and oversight, or to the public for the purposes of accountability and transparency. Similarly, citizens and businesses spend time and resources responding to public data collections, and when governments are not using the data effectively and efficiently it can frustrate the public that was burdened by data collection in the first place. While there are multiple reasons for this gap, one major acknowledged area of focus for improved use is the workforce itself. The Organisation for Economic Co-operation and Development (OECD) identifies data literacy as an [3] essential skill for public sector innovation. There has been increased discussion of data literacy at the United Nations in recent years.[4] Countries around the world are beginning to recognize the value a data literate workforce offers for the public sector. These efforts also recognize the importance of data literate organizations. There have been efforts in the United Kingdom, Australia, and the U.S., among others.[5]

The focus on data literacy from national governments and the UN and OECD is not an accident; it is based on increased emphasis in the private sector (see “Data Literacy Lessons from the Private Sector”), strong examples in some countries (see “Emphasizing Data Literacy in the United States” box), and a growing body of research to encourage the public sector to follow suit. The goal of these endeavors for the public sector arises from a motivation that varies slightly from the private sector, and also affects the design and approach used. For example, governments focusing on data literacy tend to not only focus on data for the sake of data, but they also strive to adapt or shift organizational culture to focus on more data- and evidence-informed decision- making or policymaking activities. The decision-making environment for most public sector organizations also means there should be an effective operational relationship between data experts and others in the organization with different specialties or job functions.

In other words, every individual in an organization can be data literate even if not a technical data expert. Aligning the abilities to understand and use data across the different types of employees and levels of expertise is essential for data literate organizations and a sustained culture that demands the use of data, applies insights from data to decision-making, and adjusts behaviors and organizational activities based on available information.

An expansive body of research exists from the last century on organizational change management, organizational improvement and learning, and accountability for public sector organizations. The research and theory from the public administration field collectively suggest that achieving a data literate organization is more complex and nuanced simply having a data literacy program, training staff on required skills, and facilitating ad hoc or organic process change.[16] Indeed, the capacity-building activities for data literate organizations needs to overcome bureaucratic inertia, sustained leadership support, allocation of resources, and iteration to continuously improve processes and operations using data. This is achieved while also recognizing that individual-level literacy contributes to the collective organization-wide goal of capacity for effectively using data.

 

For public sector organizations, the practical benefits of data literacy clearly align with more popularized concepts like evidence-informed policymaking and evidence-based practice. Data literate individuals and organizations collectively seek continuous improvement achieved in two optimal stages. The first desired stage is single-loop learning where data and insights are used to identify, then correct, a single issue. Individuals across organizations may use data for a wide range of operational or real-time decisions that address simple or one-time fixes. This is typically an intermediate stage of evolving into a fully data literate organization. The second and more desirable stage is double-loop learning, where not only do single decisions benefit from using data, but also changes are made to organization strategy, policy, and practice to reflect insights from data.

One major difference in applying learning models and data literacy capacity-building practices between the public and private sector organizations is what single- loop learning means for achieving desired outcomes.In the private sector these instances of using data can dramatically improve efficiency of program operations that result in decreased costs and increased revenues which generate an increasingly positive return on investment (ROI) for business. If the ROI is measured in dollars and other outcomes are not quantified, then the private sector firms achieved a relatively straightforward measurement strategy.

In the public sector, while spending and efficiency goals are generally appropriate, measured outcomes typically focus on effectiveness of program implementation and the scaled impact on numerous metrics which may not be easily or rapidly quantified. For example, a national or regional public health agency may choose to regulate air pollution for specific categories of emissions. While the health agency can monitor emissions (the “output”), the ability to assess long-term health effects requires a more significant strategy for collecting relevant, reliable, and valid data about health outcomes. Outcomes that are often the mission or goal of government programs can be distinctly more difficult to measure based on scale, geographic dispersion, time, and clarity than simpler metrics from accounting systems reflecting revenues, expenditures, and productivity.

The measurement of data literacy at the individual and organizational levels has moved in recent years from conceptual to more detailed analytical approaches. A data literacy index applied globally to the private sector offers opportunities for companies to benchmark to regional competitors and firms around the world. The index includes information on organizational performance with employee data skills, decision- making processes, and dispersion of data use in the organization.[17] The Open Data Institute’s conceptual Data Skills Framework also offers opportunities for businesses and public sector organizations alike to reflect on balancing technical skills with governance and leadership factors.[18]

Few examples of public sector measurement tools exist in the public domain, although numerous frameworks are in circulation. The PoliVisu survey instrument was pilot tested in cities in the European Union, specifically collecting information about the use of big data, and lists competencies for determining the degree of proficiency.[19] One that was developed as part of the U.S. Federal Data Strategy presents a “playbook” model for identifying critical skills, assessing gaps, conducting organizational analysis, and planning for improvements. While the playbook offers a general conceptual model for measurement, it stops short of developing a consistent tool for public sector organizations to use in practice. In parallel, there are some existing government surveys in the U.S. that include relevant questions, such as the Federal Employee Viewpoint Survey conducted by the Office of Personnel Management and the Government Accountability Office’s annual Federal Managers Survey.

At the individual level, literacy is often measured through assessments or surveys. For example, one instrument applied to literacy for educators poses scenarios that provide context, data, and other information to determine how the teachers would respond to the information.[20] Another instrument poses conceptual questions about individual-level comfort with data and statistical concepts to categorically rate respondents.[21] Several frameworks also exist that were explicitly designed for non-governmental organizations, but could be adapted for public sector contexts.[22] The Data Lodge uses the proprietary Data Literacy Program Maturity Model and Assessment for assessing baseline information and measuring progress throughout the data literacy programming.

As the data literacy field matures in coming years, the measurement will likely become an area of increasing emphasis for evaluating data literacy programs.This means improved tools for organizational- and individual-level literacy relevant to the public sector will also likely need further attention and development.

Public Sector Data Literacy in Practice: Examples of Pioneering Programs in the U.S.

While the measurement and tools for assessing data literacy at the organizational level in public sector organizations continues to develop, there are clear examples of early adopters or “pioneers” in initiating data literacy programs in the U.S. These pioneers navigated the challenges of launching, testing, and in some cases redesigning data literacy programs and strategies. Each contributes to lessons about data literacy programs that are relevant to governments at all levels around the world, though we caution about overgeneralizing the U.S. experience without appropriate and necessary adaptation to the unique contexts other countries and governments experience.

Federal Government Examples

• Air Force. The United States Air Force prioritized developing a data literacy program under the leadership of CDO Eileen Vidrine, recognizing benefits for using data to improve individual decision-making across the service’s units.[23] While the Air Force approach prefers the term “data fluency” and aligns with “data acumen” the overall approach is a long-term effort to align with the service’s mission and the core value of “excellence in all we do.” The Air Force’s data literacy program focuses on the experience of individual airmen as a tactical approach, while strategically adopting emerging technologies that are appropriate for organizational data maturity.

“We started small and took an agile approach where we consistently brought out more capability every two to three weeks and people could see it maturing in front of their eyes and they could see value in it and that helped create more change champions for us.”

—Eileen Vidrine, CDO, Secretary of the Air Force [24]

The Air Force organizationally improved data literacy through iteration and small-scale changes. The Air Force CDO’s approach was to attract buy-in from a cohort of senior leaders at the Air Force, who could also act as change agents across the organization in an agile approach. Along with incorporating a methodology like development security operations (DevSecOps), the Air Force could demonstrate continuous progress highlighting data maturation and new analytical capabilities to others in the organization. The strategy also allowed the CDO and other senior officials to more clearly communicate about the value of using data with relevant examples.

“It’s really about [Air Force] leaders investing and encouraging their teams to really invest in themselves to really accelerate the innovation.”

—Eileen Vidrine, CDO, Secretary of the Air Force [25]

By having leadership at varying levels embrace data literacy, the U.S. Air Force has many efforts and pilot programs expanding and increasing their data literacy, with some tangible benefits. The Air Force partnered with both the Air Force Institute of Technology and the Air Force Academy to create data science programs for cadets, encouraging these future leaders to pursue programs that directly correlate with data literacy.[26] The Air Force also launched less time-intensive datathons, where commanders from the field would pose real-world problems with the goal of having participants work together and solve it using data. These datathons helped demonstrate how data use can address practical issues faced in the field, while also identifying possible solutions that can be adopted into practice.

  • Education Department. The U.S. Department of Education, under the leadership of CDO Greg Fortelny published a comprehensive data strategy in 2020. The strategy outlines a broad framework for enhancing data governance and the use of data across the department. The strategy recognizes that coherent data policy is foundational, aligns with the data lifecycle, and enables data literacy and skills development across the agency. The Education Department’s data strategy includes a specific goal about building the workforce’s capacity to leverage data. The goal specifically defines literacy across a continuum, from universal and applied data literacy practices to advanced data professional skills and mastery. This includes definition of competencies and proficiencies, integrated with career maps and development resources for the entire workforce. Implementing a data literacy program at the Education Department is explicitly identified as a key objective in the data strategy.

    Launched in June 2021, the Education Department’s Data Literacy program establishes a shared language across different roles and offices. The program promotes essential awareness in four areas of relevant skills: evidence, analytics, visualization and decision-making. Training is complemented by sessions across these four pathways. Weekly agency- wide forums provide microlearning, developed, and presented by department staff from multiple offices, with facilitated discussion. The program engages office ambassadors, now in place across all major, programmatic grantmaking offices, to reach the 4,000 staff, customize materials for enhanced relevancy and support staff in asynchronous learning.

  • Census Bureau. In the U.S. the activities of the Census Bureau focused on collecting, managing, and using data since the first decennial census in the late 18th century. Data literacy is widely viewed as a foundation within the Census Bureau’s culture, and while it emerges often in the similar concept of “statistical literacy,” the applications and lessons are key. All Census Bureau staff routinely participate in workshops and training sessions about the data assets produced by the bureau and, importantly, the ethical and legal requirements for protecting Census- managed data. The Census Bureau is in a unique position of focusing on both internal and external literacy, with the publication of major data resources for public dissemination. In the case of the Census Bureau, like other federal statistical agencies, some of the expectations for how data literacy activities occur are also specified in law such as with the data protection and confidentiality requirements. The Census Bureau’s Opportunity Project focuses on a wide range of topics that include data accessibility and use through improved technological applications.[27] For example, one team in 2021 is developing a tool to encourage data-literacy with Census data for teachers.[28]

    Following a needs assessment and skill gap analysis in 2019, the Census Bureau launched a Data Science Training Pilot in 2020 which aims to upskill existing employees with technical backgrounds and to engage Census staff about emerging data science capabilities over a six-month period.[29] Census relies on vendor support as well as in-house experts, and requires participants to apply skills with capstone projects.

  • Centers for Disease Control. With more than 10,000 employees, the Centers for Disease Control (CDC) at the Department of Health and Human Services (HHS) launched a data science upskilling pilot in 2019 for 80 employees, and a second cohort in 2020. Multiple vendors provide the training materials for CDC and participants have an opportunity to apply skills through a capstone. In 2020, the first cohort of participants presented their projects to an agency- wide forum. The initial learning from CDC’s upskilling project identified a need for locating and securing mentors early, and to facilitate greater collaboration across CDC with relevant stakeholders. These activities were underway even before CDC hired its first CDO in 2020.[30]

    CDC is also unique among federal agencies in that within CDC is the National Center for Health Statistics (NCHS), one of the largest federal statistical agencies in the U.S. NCHS offers an extensive set of training and instructional activities that support CDC staff and partners in collecting and managing high- quality data.

  • Treasury Department. The U.S. Department of the Treasury’s 100,000 employees were introduced to a scaled Data Literacy Program in 2020, following a successful pilot at the Internal Revenue Service (IRS).[31] The program is an online, virtual system to provide learning resources for beginning-level sessions on data visualization, big data, decision- making, and analytical literacy. This was the first-ever data literacy program attempt for the entirety of the Treasury, through a $1.1 million vendor contract.32 The program is facilitated by the Chief Human Capital Officer of the department, which notably designated five officials to rotate as the department’s CDO through early 2020, when the Treasury Department designated a single interim CDO. The Treasury Department scaled its virtual data literacy program before a needs assessment was conducted at the agency, though this step largely occurred because the IRS had piloted and contributed resources to an integrated program. Among the cited lessons learned from the scaling experience at Treasury is the need for a self-paced program, the importance of assessing what existing training opportunities already exist for employees, and the role of collaboration with leadership in the department and other stakeholders to garner support and marketing strategy for the effort.[33]

    • Federal Deposit Insurance Corporation. The Federal Deposit Insurance Corporation is one of the independent financial regulatory institutions in the U.S. In 2020, under the leadership of CDO Jacques Vilar, the FDIC conducted a survey-based needs assessment of its workforce. The results of the assessment led to a gap analysis and the formulation of a performance plan to address gaps in mid-2020.34 Most FDIC employees were intermediate to advanced literacy levels according to this assessment, and the new training curriculum developed recognized different skill sets and needs across the agency. That curriculum was developed alongside existing human capital training plans, to launch an 18-month pilot program, which is slated to inform the scaled program in 2022. Early lessons from the FDIC include the need to assess the entire workforce’s literacy, recognizing the different training and skills for various roles at the agency, the need to gather buy-in from senior leaders, executives, and employees across the agency, and that existing training resources can be leveraged and adapted to enhance data literacy. [35]

  • State Department. The U.S. State Department leads the nation’s foreign policy through diplomatic efforts, advocacy, and foreign aid, managing nearly 70,000 employees domestically and internationally. In September 2021, through the leadership of CDO Matthew Graviss, the State Department issued its first Enterprise Data Strategy, aiming to leverage “data as a critical instrument of diplomacy.”[36] Within the strategy, State acknowledged the varying levels of data literacy among the workforce and determined the strategy’s primary objective “to prepare its workforce for an increasingly data-driven world... [by investing] in data fluency initiatives to enhance data and analytics skills at all levels and across the enterprise.”[37]

Global events demonstrate that, now more than ever, the department’s workforce needs timely, data-informed insights to make key mission and management decisions...Our approach is mission-oriented, which allows us to bolster data and analytics efforts toward the department’s top priority issues.”

—Matthew Graviss, CDO, U.S. State Department

The first goal of the strategy is to cultivate a data culture, which first requires strengthening data fluency (i.e., data literacy). The State Department invests in data training activities through the Foreign Service Institute, where more than 2,200 employees benefit from the data fluency program.[38] Through the CDO’s Center for Analytics, the State Department has partnered with Deloitte and other organizations, including research universities, to develop and drive the Department’s Enterprise Data Strategy forward. This clear, up-front commitment driven by the Department’s CDO demonstrates that the State Department understands the importance of building a data culture early in efforts to engage the entire organization in using data. The State Department’s top official also publicly discusses the importance of using data to the organization’s mission as well as the importance of the agency data strategy.[39]

As Graviss noted, “global events demonstrate that, now more than ever, the department’s workforce needs timely, data-informed insights to make key mission and management decisions...Our approach is mission- oriented, which allows us to bolster data and analytics efforts toward the department’s top priority issues.”

“To use data as a strategic asset, we must first empower the Department’s greatest resource: its world-class global workforce...We must continue to invest in recruiting, training, and equipping our teams to be skilled data consumers and users. This will take time, but we have none to waste.”

—Brian McKeon, U.S. Deputy Secretary of State [40]

State and Local Government Examples

  • Texas. The approach that the State of Texas took to increase data literacy within its workforce began with gaining an understanding of where their individual state agencies are in the data management journey.[41] In 2021, the Texas legislature passed legislation to create Data Management Officers across state agencies, charging them to coordinate with the agency CDO.[42] The new data management officials in Texas are helping to spread data-literate personnel across agencies, increasing overall literacy and improving statewide data practices and proficiency for using data, communicating about data, and making decisions using data. [43]

“When we talk about bringing in a Data Management Officer for an agency to restructure and refocus on how data is being looked at, data literacy is key to help facilitate change management since this is a change management opportunity.”

—Ed Kelly, CDO, State of Texas

Led by the existing Texas CDO office that organized in 2016, Texas recognized the need to foster collaboration with other agencies by sharing successes, insights, and lessons learned with each other.[44] For example, programs such as the literacy training module were developed to be leveraged within other agencies to increase the overall impact of the training and share success across government organizations. Officials in Texas also created products that would help integrate data literacy within the agency such as briefs on data literacy and data management. Texas also had success with the use of promotional materials and small video to build data literacy programs in the state. According to Texas’s CDO, personal responsibility for data literacy is important: state leaders “know that this is their responsibility...everyone needs to be involved—the Chief Data Office and collaboration across the enterprise, including external stakeholders.”

“There are individuals who think if it is data, then its IT and business area doesn’t really need to be involved with it, but changing and shifting that paradigm helps with education and literacy so people aren’t so fearful of the issues and change that is coming with the new way to look at information and data.”

—Ed Kelly, CDO, State of Texas

  • Baltimore, Maryland. The city of Baltimore initiated a change over the course of decades that implemented the CitiStat program, aligning performance management activities with statistical analyses to present information for decision- makers in dashboards. City officials routinely meet to discuss problems of the day with performance information, but expect the program managers to explain key indicators and propose solutions.[45] In practice, the CitiStat efforts built a culture across the city government for using data more effectively in identifying and addressing problems. In 2020, Baltimore was recognized by the What Works Cities initiative for its incorporation of data practices to inform decision-making and also to educate the public.[46] While the practices were not explicitly referred to as a data literacy program, the initiative has many of the key features that support innovation and use of data in public sector organizations.[47]

    • San Francisco, California. The city of San Francisco began its work to leverage data as a connecting and empowering force over ten years ago. DataSF, the City’s open data and data activity hub, houses data services for city departments. On the hub, city employees can submit and publish datasets to share, employ data visualization tools to help make data-informed decisions, and access data science resources, like advanced analytics and applied statistics.[48] The City of San Francisco also maintains a Data Academy for staff, launched in 2013. Since then, nearly 6,000 attendees have taken advantage of the data courses, training, and workshops to improve data skills and knowledge, or data literacy.[49] Each of the listed examples highlights an agency or program that public sector executives and C-Suite executives could consider as an applicable model in their unique contexts.

Success Factors from Data Literacy Pioneers in the U.S.

Building on past research and exploring the early pioneers for the public sector described above, there are clear patterns that emerge for success in these far-sighted organizations. As is common with any emerging field, varying definitions and interpretations of “data literacy” are prevalent, which can affect program design. Being explicit in what problems are being solved for, as well as the needs and drivers to be addressed with a data literacy program, are vital to mitigate false starts. Agencies should formulate a case for change that incorporates these lessons. Among the successful agencies and programs reviewed, factors and patterns for organizations moving rapidly to become data literate organizations include clear leadership roles, conducting a gap analysis, common language, data access, prioritization of data governance, and the use of data in decision-making.

Engage Senior Leaders with Clear Roles and Expectations

The CDO of a public sector organization is in charge of establishing the vision and strategy to maximize use of the organization’s data resources to achieve its objectives, including with ongoing efforts to bolster data literacy. Yet public sector organizations without a CDO can still launch and implement successful data literacy programs. A key factor for the success of any such program is that senior leaders and executives in the organization support the implementation and use of staff time for improved literacy. Without such buy-in, literacy programs may be successful at training and upskilling employees who find the organizational culture and the actual use of data underwhelming or frustrating.

For organizations with CDOs, that individual is the likely champion to encourage data literacy efforts. This was the case in nearly all of the examples provided in the last section, such as the U.S. Education Department, State Department, and the state of Texas. The CDO can acquire buy-in from executive leadership to establish or bolster a data culture that cuts across the entire organization. The buy-in can also be critical for working across organizational silos and even in formulating long- term collaborations with the human resources units that may have resources allocated for training activities within current budget levels.

Senior leaders empower the workforce to act on the data—especially when it shows that a change of direction is needed to improve. Executive leaders in organizations also set the tone for how data and evidence can be useful for decision-making, while also fostering environments conducive to continuous learning, improvement and innovation, and incentivizing or rewarding employees who seek to use data to improve processes, productivity, and outcomes. It is important for those executives and other leaders to know they have a role and to delineate responsibilities for data literacy programs that may fall to others beyond the CDO.

While senior leaders have an important role to play in building capacity and encouraging data literacy, successful programs require individuals at every level of the organization to participate and engage in tandem.

Clarify Target Competencies and Personas for Actionable Gap Analysis

When fostering data literacy within a public sector organization, far-sighted CDOs start with the identification of core competencies that are needed, then assess the workforce’s data literacy relative to those competencies while recognizing different types of employees’ job responsibilities and functions. Some organizations, such as the FDIC, choose to survey the entire organizational staff, while others choose a sample of employees to support formulating an effective strategy.

Many data competencies are cross-cutting, such as ensuring that every employee knows obligations for ethical use, privacy protections, and cybersecurity protocols. But for many topics the literacy training can be tailored to different workforce segments based on their individual needs. Public sector organizations with a data-driven culture embrace a diversity of roles and skills in the workforce. Instead of relying on siloed teams of highly technical quantitative experts, they cultivate a wide variety of people throughout the organization who are curious and capable of translating methods and meaning in the context of the organization’s activities.

Next, organizations assessing the data literacy of the workforce should conduct a gap analysis to prioritize major areas or priorities that are otherwise unaddressed. This activity should be conducted as a collaboration between the CDO, the human resources office, and the organization’s evaluation staff.

After identifying competencies, surveying employees, and analyzing the gaps, organizations can develop a plan for how to upskill workers and meet critical needs. Pilot testing the data literacy program with a smaller portion of the staff before scaling can also help strengthen the curriculum and community before operating at scale.

Whatever the type of data literacy program—whether a training program or a holistic approach—continuous assessment of effectiveness and shortcomings should be conducted. Data literacy programs should have their own measurable performance metrics and routine program evaluations to identify areas for ongoing improvement and learning.

Cultivate a Common, Shared Language

Talking about data in any organization can be challenged by disciplinary and ontological differences that emerge in college or graduate school for many. Developing a shared language is one way of strengthening the culture for discussing and using data in an organization. Employees in data-literate public sector organizations intentionally foster and curate a shared vocabulary that underpins how they talk about data and how they connect operational and data concepts. They use that vocabulary in the context of everyday interactions, in meetings with executives and the public, and as a basis for outcome-oriented decision-making.

In the private sector, many data analytics projects fail because they are not aligned with the business strategy. The lack of a shared language is likely a major aspect for why those developing strategy and those discussing the analysis may be misaligned. This same concept is relevant for public sector organizations where data analysts, statisticians, and evaluators may be producing relevant insights but discuss the results of projects in a manner that is inaccessible to key executives and program managers. Development of a common vocabulary can happen organically over time, but it should be a deliberate, systematic initiative if it is to span the whole enterprise, not just the data experts at the organization. Developing a common language for communication can also support the development of a community of practice where individuals collaborate more openly to address key analytical needs, shifting the reliance on a few experts for support to a broader community for engaging to use data for decision-making.

Improve Data Accessibility

Organizations need accessibility to underlying data and relevant information to encourage the use of data. Greater data accessibility was an underlying characteristic in nearly all examples from the previous section. This is not to say that access to data is a free- for-all in data literate organizations, but rather that organizations appropriately applied access restrictions based on data sensitivity and calibrated the access to certain types of data based on purpose, need, and training. Public sector organizations manage extensive datasets with personally-identifiable information and confidential business information, which must be protected according to pledges made during collection and based on applicable laws and regulations.

When organizations implement tiered access systems for enhancing data accessibility to employees and stakeholders, it helps ensure that data are accessible to analysts beyond the few specialists with the most expertise in an organization. Public sector organizations, working through the CDO, may choose to survey executives and staff about their knowledge of existing data assets and whether they have access to critical data for decision-making. While the survey would identify some knowledge about perceived gaps, results may also bias towards knowledge of the data that exist.

In the private sector, while 80 percent of executives indicate they have access to key data assets to be successful in their positions, just 67 percent of middle managers indicate the same.[60] The same could very well be the case in public sector organizations. For this reason, public sector organizations should maintain accessible data inventories with appropriate
metadata to explain core data elements that can be used as well as any information about limitations to access, a key requirement of the OPEN Government Data Act. Knowledge of what data exist can also expand organizational understanding for how to apply data assets to answer key questions for decision- makers and to evaluate programs in response to accountability initiatives.

Align Data Governance and Data Literacy

Data governance processes within an organization should align with the goals of a data literacy program. Data governance focuses on the roles, responsibilities, and processes for ensuring accountability for data, the ownership of data assets, and the capabilities for responsible data sharing and use. Arkansas is an example of a state in the U.S. that recently established
a data governance committee, as have most federal agencies in the U.S. with the committees typically chaired by the CDO. In cities like San Francisco, the alignment between training and skills development with staff submissions of open data offers an applied approach to aligning literacy with governance practices.

Key governance goals typically include minimizing risk to data owners and subjects, establishing internal rules for data use, implementing compliance requirements for access and use, improving internal and external communication about data, facilitating access to data, and ensuring continued existence of the company through risk management and optimization.[61]

These are all core features that can be included in a comprehensive, enterprise-wide data literacy program. This alignment may occur naturally as part of the data literacy assessment and skills gap analysis, but should also be a deliberate area of emphasis.

Encourage the Use of Data in Decision-Making

The productive use of data is ultimately the goal of data literacy programs, data governance, and even the CDO position. Yet, decision-making processes in public sector organizations are often opaque even to those working in government.[62] The pioneering agencies and cases in the previous section typically engaged specifically on improving the transparency of decision- making processes or otherwise communicating about how data and evidence were used in critical actions or policies. For example, in Baltimore the use of CitiStat aligned the need for employees across the workforce to engage with data literacy while clearly demonstrating how the data informed decisions.

In the U.S. the alignment of these types of needs and the availability of data resources may shift for many federal agencies as evidence-building plans, or learning agendas, are produced to better articulate knowledge needs and gaps for major decisions that are operational or policy-oriented.[63] While some processes and decisions will likely always be deemed deliberate, senior leaders and executives can facilitate the shift in organizational cultures by also demonstrating where they are using data for decision-making, or at least where it would be helpful if not available.

Recommendations for Prioritizing and Sustaining Data Literacy Programs in Government

Extending from the success factors identified above, his report concludes with seven recommendations relevant to public sector organizations in the United States, regardless of whether they are at the national, state, or local level. Each recommendation will need to be tailored to the appropriate context but can help set the stage for prioritizing and sustaining data literacy organizations in government.

Recommendation #1 → Sponsorship: Agency heads should designate a CDO or an executive leader
with responsibility for establishing a data literacy program. While successful data literacy programs can occur under the auspices of various officials or units in a public sector organization, an official should be designated to be responsible for ensuring the effort happens. The role of these designated leaders is to build support for the data literacy program, market efforts effectively to internal stakeholders, assess skills and gaps, to identify quick wins, and to develop, implement, and assess the program. This role may be natural for the CDO in an agency and should be conducted in partnership with human resources staff.

Recommendation #2 → Resource Allocation: Agency heads should allocate resources for assessment of data needs and training. At the outset of launching a data literacy program, some level of basic resources and staff time will be needed. For example, if a data literacy program includes a training or upskilling component, unit heads and program managers must be willing to allocate some of their staff’s time to participate in training and educational activities. Senior leaders also have the capability to articulate a priority and ensure a basic level of resourcing is available. While no set level of resources has been identified as levels will be unique to each organization’s needs and existing levels of data literacy. Resources, however, should generally expand over time as the quick wins prove value and the workforce becomes increasingly data literate. Resources in this context include strategies for bolstering engagement, promoting and encouraging the community of practice, and even periodic refresher courses on methods and privacy protection protocols.

Recommendation #3 → Competencies, Personas, and Curriculum: Agencies should collect survey data on employee data skills, needs, and use. As data literacy programs are being developed, it is critical that the needs assessment and skills gap be conducted to ensure the competencies relevant for a program or organization align with projected needs. This information could be collected through a survey instrument that also asks for examples of data use to bolster the list of relevant examples. In the U.S., for example, this could be incorporated with questions on the Federal Employee Viewpoints Survey to monitor literacy over time and support evaluation of literacy programs.

Recommendation #4 → Learning Agenda: Agencies should develop a learning agenda as an initial step
in articulating data needs, coordinated between the evaluation official and the CDO. Beyond the senior executive in public sector organizations, increasingly the learning agenda is a strategy for demonstrating humility in data needs while also presenting the workforce and partners a clear signal about what is needed for future decisions. Learning agendas should be increasingly encouraged in public sector organizations, including at a division or operating level where the questions and data needs can be tailored to best achieve organizational goals. Publication of a learning agenda should occur through a participatory process with feedback and engagement from a range of stakeholders. A key ingredient for a successful learning agenda that can support the data literacy program as a whole is that the effort is not conducted solely to comply with law but rather because it embodies the learning ethos of the organization.

Recommendation #5 → Teaming for Leverage: Agency senior leaders should encourage public- private partnerships in their data literacy programs. Even with incredible expertise in many public sector organizations, establishing public-private partnerships is a strategy for benefiting from expertise and resources in universities, the private sector, or non-profit organizations that can support emerging communities of practice and upskilling of the workforce.

Recommendation #6 → Transparency: Agency senior leaders should be transparent and informative in describing valuable data uses. Senior leaders in the organization can set the stage for motivating culture change and individual employees participating in data literacy programs by simply discussing the role data plays in their decision-making activities. For executives at the top-level of public sector organizations to describe how they are using data to their employees and stakeholders, including with specific examples, will demonstrate that a demand for data exists at the top of the organization and establish an expectation that it should exist at all levels of the organization. If executives are also able to offer an honest assessment of data needs and gaps, employees may also be incentivized to offer suggestions or approaches for addressing those gaps. While top-down leadership is not the only way to establish and sustain a data literacy program, the leadership can demonstrate the value and establish consistent demand for the workforce to meet the need. Sharing strategies for using data could occur through multiple forums, such as a town hall with agency staff, feedback on decision memos, or even public statements in regulatory publications.

Recommendation #7 → Incentives: Agency senior leaders should incentivize the use of data through policy, grants, and guidance. With a more data literate workforce, agencies should increasingly be able to accommodate efforts to incorporate expectations about data use in relevant activities, such as policy implementation, grant deliverables, and guidance documents. Incentivizing the use of data may mean that certain traits or characteristics receive preference points in competitive grant programs, or that grantees are required to report on outcomes and also document methods for reporting such data. These incentives may even align with broader efforts to encourage evidence-informed policymaking in public sector organizations by recognizing and linking to program evaluation initiatives.

Perhaps one of the great but largely unstated benefits of improving data literacy in public sector organizations around the world is that government employees can then be better prepared to facilitate and encourage data literacy for constituents, beneficiaries, and users of government services.[64] This, in turn, can empower the citizenry to hold their government institutions accountable and to encourage evidence-informed policies and practices.

Ultimately, data literacy is all about shared language and a community—and that starts with a shared definition and framing of data literacy itself. This report provides a shared definition, explains why data literacy matters, then explores multiple examples of data literacy programs to identify key success factors.

There are many considerations for how to build and sustain an effective data literacy program. The most important one: Every organization needs to start somewhere. Recognizing that program development is a learning process and that it will evolve over time also means that no agency or organization should aspire for a perfect program at the outset; every program will take its own course. What is essential is that every organization starts to build its capacity for using data and evidence—and that all begins with data literacy.

ENDNOTES

  1. Logan, V. Data Literacy Programs 101. Washington, D.C.: Data Foundation and The Data Lodge, 2020. Available at: https://www.datafoundation.org/events-list/data-literacy-programs-101-the-what-why-and-how-to-get- started/2020; The Data Lodge. Data Literacy? What’s in a Name? The Data Lodge, 2021. Available at: https://www. thedatalodge.com/blog/webinar-data-literacy-whats-in-a-name; the definition varies slightly from others yet maintains conceptual alignment, such as with the definition offered by Wolff, A., D. Gooch, J.J. Cavero Montaner, U. Rashid, and G. Kortuem. 2016. “Creating an understanding of data literacy for a data-driven society.” The Journal of Community Informatics 12 (3): 9–26.

  2. Logan, 2021.

  3. Organization for Economic Cooperation and Development (OECD), Skills for a High Performing Civil Service. Paris: OECD Publishing, 2017. Available at: https://read.oecd-ilibrary.org/governance/skills-for-a-high- performing-civil-service_9789264280724-en

  4. Rogers, K. and Peterson, L. How to Build Data Literacy at the UN: Lessons from OCHA. New York: United Nations, 2019. Available at: https://unite.un.org/blog/how-to-build-data-literacy-at-the-UN-ocha

  5. Bonikowska, A., Sanmartin, C., and Frenette, M. Data Literacy: What it is and How to Measure it in the Public Service. Analytical Studies: Methods and References. Canada: Statistics Canada, 2019. Available at: https:// www150.statcan.gc.ca/n1/pub/11-633-x/11-633-x2019003-eng.htm; OMB. Federal Data Strategy 2020 Action Plan Progress. Washington, D.C.: White House, 2021. Available at: https://strategy.data.gov/progress/; UK Department for Digital, Culture, Media, & Sport. National Data Strategy. London, United Kingdom: UK Government, 2020. Available at: https://www.gov.uk/government/publications/uk-national-data-strategy/ national-data-strategy.

  6. Data Foundation. Future of Open Data: Maximizing the Impact of the OPEN Government Data Act. Washington, D.C.: Data Foundation and Deloitte, 2019. Available at: https://www.datafoundation.org/future-of-open-data- maximizing-the-impact-of-the-open-government-data-act.

  7. Office of Management and Budget (OMB). Federal Data Strategy – A Framework for Consistency (M-19-18). Washington, D.C.: White House Office of Management and Budget, 2019. Available at: https://www.whitehouse. gov/wp-content/uploads/2019/06/M-19-18.pdf.

  8. Data Foundation. CDO Insights: 2021 Survey Results on the Maturation of Data Governance in U.S. Federal Agencies. Washington, D.C.: Data Foundation, 2021. Available at: https://www.datafoundation.org/cdo-insights- report-2021.

  9. Data Foundation, 2021.

  10. OMB. Federal Data Strategy 2020 Action Plan Progress. Washington, D.C.: White House, 2021. Available at: https:// strategy.data.gov/progress/.

  11. OMB. 2020 Action Plan. Washington, D.C.: White House, 2020. Available at: https://strategy.data.gov/2020/action- plan/.

  12. Ward, N. The State Chief Data Officers Network. Washington, D.C.: Georgetown University, 2020. Available at:

    https://storymaps.arcgis.com/stories/2ebdaee487c54ff98312ce664bf7b76f

  13. Groves, P. Arkansas Hires CDO in Partnership with University System. Folsom, California: e.Republic/ Government Technology. 2020. Available at: https://www.govtech.com/people/arkansas-hires-cdo-in- partnership-with-university-system.html.

  14. Pattison-Gordon, J. Maryland Gov. Creates State Data, Privacy Officer Positions. Folsom, California: e.Republic/ Government Technology, 2021. Available at: https://www.govtech.com/workforce/maryland-gov-creates-state- data-privacy-officer-positions.

  15. Management Performance Hub. Data Proficiency Program. Indiana: Indiana.Gov, 2021. Available at: https://www. in.gov/mph/data-proficiency-program.

  16. See summary of relevant literature in Chapter 2 of Hart, N. Evaluation at EPA: Determinants of the U.S. Environmental Protection Agency’s Capacity to Supply Program Evaluation. Washington, D.C.: George Washington University, 2016.

  17. Qlik. The Data Literacy Index: Results Summary. Available at: https://www.qlik.com/us/-/media/files/resource- library/global-us/register/analyst-reports/ar-the-data-literacy-index-en.pdf.

  18. Open Data Institute. Data Skills Framework. London, United Kingdom: ODI, 2020. Available at: https://theodi.org/ article/data-skills-framework/.

  19. Bonikowska et al, PoliVisu Data Literacy Survey. Horní Bříza, Czech Republic: Plan4all, 2018. https://www. plan4all.eu/2018/02/polivisu-data-literacy-survey/ and European Commission. PoliVisu. Available at: https:// cordis.europa.eu/project/id/769608/reporting

  20. WestEd. Developing Assessments of Data Literacy. San Francisco, California: WestEd. Available at: https://www. wested.org/project/data-literacy-assessment-development/#

  21. Qlik. Data Literacy Assessment. Available at: https://thedataliteracyproject.org/assessment

  22. For example: https://scholarspace.manoa.hawaii.edu/bitstream/10125/50519/paper0632.pdf

  23. Information summarized from presentation at the MIT CDO IQ in 2021.

  24. Disclaimer: These views represent those of the individual and do not necessarily represent the view of the Department of Defense or those of the United States Air Force.

  25. Disclaimer: These views represent those of the individual and do not necessarily represent the view of the Department of Defense or those of the United States Air Force.

  26. Williams, L. USAF primed to launch new phase of data strategy. McLean, Virginia: FCW, 2020. Available at:

    https://fcw.com/Articles/2020/11/30/vidrine-usaf-data-strategy.aspx?Page=2

  27. U.S. Census Bureau. The Opportunity Project. Available at: https://opportunity.census.gov/sprints/.

  28. Tamez-Robledo, N. Can Data Literacy be Fun? The Census Bureau is Powering a (Student-Built) App for that. EdSurge, 2021. Available at: https://www.edsurge.com/news/2021-11-11-can-data-literacy-be-fun-the-census- bureau-is-building-an-app-for-that

  29. Federal Chief Data Officer Council’s Data Skills & Workforce Development Working Group. Data Skills Training Program - Case Studies. Federal CDO Council, 2021. Available at: https://resources.data.gov/assets/documents/ CDOC_Case_Studies_Final_v3.pdf

  30. Vincent, B. During Pandemic, CDC Aims to Hire Chief Data Officer. Nextgov, 2020. Available at: https://www. nextgov.com/analytics-data/2020/04/during-pandemic-cdc-aims-hire-chief-data-officer/164330/

  31. Federal CDO Council’s Data Skills & Workforce Development Working Group. 2021.

  32. USASpending.Gov. Available at: https://www.usaspending.gov/award/CONT_

    AWD_2032H820F00084_2050_47QTCA19D002B_4732.

  33. Federal CDO Council’s Data Skills & Workforce Development Working Group. 2021.

  34. FDIC. FDIC Federal Data Strategy Annual Report. 2021. Available at: https://www.fdic.gov/about/strategic-plans/

    data/fdic-federal-data-strategy-annual-report.pdf

  35. CDO Council’s Data Skills & Workforce Development Working Group. 2021.

  36. Department of State. Enterprise Data Strategy: Empowering Data Informed Diplomacy. 2021. Available at: https://

    www.state.gov/wp-content/uploads/2021/09/Reference-EDS-Accessible.pdf

  37. Department of State 2021, p. 12.

  38. Heckman, J. State Dept. CDO makes diversity top management priority under data strategy. Federal News Network, 2021. Available at: https://federalnewsnetwork.com/big-data/2021/11/state-dept-cdo-launches-6- month-sprints-as-part-of-agency-data-strategy/

  39. Blinken, A. Secretary Antony J. Blinken at the National Security Commission on Artificial Intelligence’s (NSCAI) Global Emerging Technology Summit. 2021. Available at: https://www.state.gov/secretary-antony-j-blinken-at- the-national-security-commission-on-artificial-intelligences-nscai-global-emerging-technology-summit/

  40. Abeyta, L. Top Government Chief Data Officers to Watch in 2022: FERC’s Kirsten Dalboe. WashingtonEXEC.Com, 2021. Available at: https://washingtonexec.com/2021/11/top-government-chief-data-officers-to-watch-in-2022- fercs-kirsten-dalboe/

  41. Department of State 2021, p. 4.

  42. Information summarized from presentation at the MIT CDO IQ in 2021.

  43. Nelson, Kolhorst, Zaffirini. Bill SB 475. 2021. Available at: https://capitol.texas.gov/BillLookup/History. aspx?LegSess=87R&Bill=SB475

  44. Bastin, M. Even the Data is Bigger in Texas. GovLoop, 2016. Available at: https://www.govloop.com/even-the- data-is-bigger-in-texas/

  45. Ash Center for Democratic Governance and Innovation. Award: CitiStat. Harvard Kennedy School, 2004. Available at: https://ash.harvard.edu/news/citistat

  46. Mayor’s Office. Baltimore City Recognized as UP-and-Coming Leader in Data-Driven Management. City of Baltimore, 2020. Available at: https://mayor.baltimorecity.gov/news/press-releases/2020-06-30-baltimore-city- recognized-and-coming-leader-data-driven-management

  47. Jacobson, M., Wavelet, M., Feldman, A. PerformanceStat: A Leadership Strategy That Keeps Innovating. Governing.com, 2019. Available at: https://www.governing.com/gov-institute/voices/col-performancestate- leadership-management-strategy-adapting-innovating.html

  48. San Francisco. DataSF. Available at: https://datasf.org/about/.

  49. San Francisco. DataSF Academy. Available at: https://datasf.org/academy/performance/.

  50. New Vantage Partners. Big Data and AI Executive Survey 2020: Executive Summary of Findings. New York: NVP, 2020. Available at: https://c6abb8db-514c-4f5b-b5a1-fc710f1e464e.filesusr.com/ugd/ e5361a_579e07bcb351420fb9c24f28237fc6f5.pdf.

  51. Qlik. The Human Impact of Data Literacy. https://thedataliteracyproject.org/humanimpact

  52. New Vantage Partners. 2020.

  53. New Vantage Partners. 2020.

  54. Qlik. The Human Impact of Data Literacy. https://thedataliteracyproject.org/humanimpact

  55. Gaskell, A. How Data Literate is Your Organization?. Forbes, 2018. Available at: https://www.forbes.com/sites/ adigaskell/2018/10/26/how-data-literate-is-your-organization/?sh=3b668bac355a

  56. Note Deloitte is the employer of one of the co-authors, a supporter of the Data Foundation, and a funder of this project.

  57. Davenport, A., Guszcza, J., Smith, T., Stiller, B. Analytics and AI-driven enterprises thrive in the Age of With. Deloitte, 2019. Available at: https://www2.deloitte.com/us/en/insights/topics/analytics/insight-driven- organization.html

  58. Qlik. The Human Impact of Data Literacy. https://thedataliteracyproject.org/humanimpact

  59. Qlik. The Human Impact of Data Literacy. https://thedataliteracyproject.org/humanimpact

  60. Qlik. Lead with Data. Available at: https://www.qlik.com/us/bi/-/media/08F37D711A58406E83BA8418EB1D58C9. ashx?ga-link=datlitreport_resource-library

  61. Olavsrud, T. Data governance. A best practices framework for managing data assets. CIO, 2021. Available at:

    https://www.cio.com/article/3521011/what-is-data-governance-a-best-practices-framework-for-managing-data- assets.html

  62. Hart, N., Yohannes M. Evidence Works: Cases Where Evidence Meaningfully Informed Policy. Washington, D.C.: Bipartisan Policy Center, 2019. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3766880.

  63. Newcomer, K., Olejniczak, K., Hart, N. Making Federal Agencies Evidence Based: The Key Role of Learning Agendas. Washington, D.C.: IBM Center for the Business of Government, 2021. Available at: https://www. businessofgovernment.org/report/making-federal-agencies-evidence-based-key-role-learning-agendas

  64. Internews Center for Innovation & Learning. Beyond Data Literacy: Reinventing Community Engagement and Empowerment in the Age of Data. MIT Center for Civic Media, 2015. Available at: https://datapopalliance.org/wp- content/uploads/2015/11/Beyond-Data-Literacy-2015.pdf

Acknowledgments

The authors thank Ashley Nelle-Davis, Lori Gonzalez, Christopher Murrell, Joe Ilardi, and Tasha Austin for their assistance drafting and editing. The authors also thank the numerous reviewers who provided invaluable feedback on earlier drafts of this paper as well as the participants and attendees of the 2021 MIT CDO IQ session on data literacy in July 2021.

Disclaimer

This paper is a product of the Data Foundation, sponsored by Deloitte and in collaboration with The Data Lodge. The findings and conclusions expressed by the authors do not necessarily reflect the views or opinions of the Data Foundation, its funders and sponsors, or its board of directors.

This publication contains general information only and the Data Foundation, Deloitte, and The Data Lodge are not, by means of this publication, rendering accounting, business, financial, investment, legal, tax, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor.

Deloitte, The Data Lodge, and the Data Foundation shall not be responsible for any loss sustained by any person who relies on this publication.

As used in this document, “Deloitte” means Deloitte & Touche LLP, and Deloitte Consulting LLP, which are separate subsidiaries of Deloitte LLP. Please see www.deloitte.com/us/about for a detailed description of our legal structure. Certain services may not be available to attest clients under the rules and regulations of public accounting.

Copyright © 2022 Data Foundation. All rights reserved.

Copyright © 2022 Deloitte Development LLC. All rights reserved.

Copyright © 2022 The Data Lodge LLC. All rights reserved.