GovQA PiPRIndex: Peers in Public Records Index
The only index quantifying FOIA for state and local government. The Peers in Public Records Index (PiPRIndex) tracks complexity over time and shows that managing requests for public records is getting harder.
Public Record Request Complexity 2018 – 2021
Navigating the PiPRIndex
PiPR Index In the News
Quantifying Complexity in Public Records
Making Sense of the Complexity Report
Calculating Public Records Complexity
PiPRIndex in the News
Since it launched in 2020, the PiPRIndex has become the standard for measuring growth in FOIA (Public/Open Records) requests for state and local government.
- GOVTECH | Exclusive: New Index Tracks Data on Public Records Requests
- ASSOCIATED PRESS | Governments delay access to public records during pandemic
- GOVTECH | Data: Why Public Records Requests Are Getting Harder to Fulfill
- POLICE1 | 6 steps law enforcement agencies should take as new laws change how police records requests are handled
- STATESCOOP | Public records requests for audio, video spiked during pandemic
- PR NEWSWIRE | Pandemic Drives Up Public Records Request Volumes While Growing Complexity Impacts Processing Time, Says New GovQA PiPRIndex Data
Quantifying Complexity in Public Records
As the Gov Tech market leader in cloud-based automated workflow (SaaS) software for governments, GovQA is singularly positioned to provide this actionable barometer for the public records industry.
Anonymized data from GovQA’s broad and diverse customer base, brings to light important trends affecting state, county and city governments on a quarterly basis.
This report, the first of the four Peers in Public Records Index reports, covers Complexity. Below we break this down by:
- Request Volume
- Quantity of Documents
- File Size
- Video Files
- OCR Files
- Clarification Emails
- User Interactions
- Total Time Spent
Q4 2021 Data: Public Records Complexity has increased 173% since Q1 2018
Request Volume
Request Volume may be the most critical quantifier of complexity as it relates to scale.
- Rapid increases in public records requests
- Request volume grew by approximately 188/month in 2018 — or 470% over the last eight years*
- Some GovQA customers getting more requests in the last 3 months than last 3 years
*2018 study from Sunlight Foundation
Quantity of Documents
Measures the scale of complexity as the total count of individual documents sent to requesters, averaged by volume.
- Each document increases the number of required interactions
- More documents = more numerous actions to complete processing
- For agencies not using software even worse
File Size
Measures the scale of complexity as the total size, in Megabytes, of all documents sent to requesters.
- Larger files = more pages, more time to review/redact
- Large files = images, audio, or video; harder to redact, more time to process
- Large files take more time to manipulate
Video Files
Measures both the complexity scale and diversity as the count of all video files inside the GovQA system
- Not all agencies manage video files
- If they do: video files = unique challenges
- Video files = largest records created/owned by agencies
- Longer to upload/download
- Difficult to redact
ORC Files
Measures both the complexity scale and diversity as the count of all OCR scanned documents and PDFs processed by the GovQA OCR robot
- These file types are not managed by all agencies; but for those that do, they introduce diversity in file types which requires unique handling.
- This metric shows how many agencies are still working with paper or files which are not readily searchable in their native format.
Clarification Emails
Measures complexity as connectivity by counting the number of clarification emails generated between requesters and those managing requests.
- Public awareness of the availability/usefulness of public records increasing
- More new requesters
- Unclear requests = more time to narrow and fulfill
- Clarification emails (and awaiting responses) increases complexity
User Interactions
Measures connectivity-induced complexity as the average number of staff involved in a request.
- Counted by unique user IDs assigned to/involved in a request
- If one person can complete processing activities themselves, request is simpler
- Higher staff count = more complexity
*GovQA user interactions trend downward as workflow automations are optimized in the software. Agencies do more with fewer staff when using automated software.
Total Time Spent
Measures the scale of complexity as the number of hours spent processing requests. Many things can affect total time:
- poorly defined requests
- voluminous requests
- large file review and redact
- multiple people involved in the process
Making Sense of the Complexity Report:
The GovQA Complexity Index is composed of two markers: the Records Marker; and the Activities Marker.
To calculate the Complexity Index, the Records and Activities Markers are weighted against Request Volume – which is the total count of public records requests received by an agency.
Records Marker:
The Records Marker is an indicator of the complexity of
the public record files themselves. It includes:
- The Quantity of Response Documents (indicates scale)
- The Sizes of Response Documents (indicates scale)
- File Type Group: Video Files (indicates scale and diversity)
- File Type Group: OCR Files (indicates scale and diversity)
Activities Marker:
The Activities Marker is an indicator of user activities
related to processing public records requests. It includes:
- Average User Interactions (indicates connectivity)
- Clarification Emails Generated (indicates connectivity)
- Total Time Spent (indicates scale)
Calculating Public Records Complexity:
Complexity, by definition, is the state of being intricate or complicated – having many interconnected parts or features. The causes of complexity as they relate to public records are:
- Scale (the number of request files in the system);
- Diversity (the extent to which request files are different); and
- Connectivity (the inter-relationships required to process requests).
These factors have been identified in the vast GovQA anonymized blind data sets and grouped for convenient analysis and use as a decision-making tool for government leaders.
To calculate complexity, we started with the data of 500 anonymized member agencies, then removed the outliers at the very highest and lowest ends of the spectrum across all metrics, leaving us with ±240 members that represented the most common experience across all 8 metrics.
Adjusted for volume and tracked quarterly, GovQA is able to quantify the predictable growth in the complexity organizations will continue to face across the country.
Complexity as a Catalyst
“Requests are increasing in complexity. Litigation throws sand in the gears of operations. It’s getting more and more difficult over time. So we must embrace new techniques and methodologies if we’re going to continue on the course of improvement that we’re all seeking here in this profession.”
– Michael Sarich, Director Veterans Health Administration, FOIA program and co-chair of the CFO Technology Subcommittee to OGIS stated at a recent FOIA Advisory Committee meeting