TL;DR
This paper introduces AuditBear, a novel automated system for analyzing electronic voting machine (DRE) audit logs to detect procedural errors and system deficiencies. By processing publicly available log data, AuditBear helps election officials identify issues like missing votes, incorrect machine behavior, and long voter lines, ultimately aiming to improve election accuracy and fairness.
What Problem Does This Paper Address?
Election officials face the immense challenge of ensuring fair and smooth elections. Electronic voting machines (DREs) generate detailed audit logs that theoretically could help uncover procedural errors and anomalies. However, these logs are currently unwieldy and difficult for election officials to use effectively for post-election audits. Manually sifting through this data is time-consuming and prone to human error. This gap leaves valuable information about election day events hidden, hindering efforts to understand what worked, what didn't, and how to improve future elections. The paper aims to bridge this gap by providing an automated, accessible tool for analyzing these logs.
Methods
The core of this work is the development of new methods for analyzing DRE audit logs. The authors implemented these methods in a web application called AuditBear. Key analytical approaches include:
- Date/Time Error Detection: Identifying discrepancies in machine timestamps, which can hinder accurate reconstruction of events.
- Vote Under-counting Detection: Comparing different log files (event logs, ballot images, system logs) to identify instances where cast votes might not have been included in the final tally. This involves checking for missing PEBs (Personalized Electronic Ballots) or PEBs that weren't uploaded.
- Long Lines Analysis: Using proxy indicators like precincts staying open past official closing times and analyzing the time between consecutive cast votes to infer periods of high voter activity and potential long lines.
- Procedural Error Detection: Identifying instances of poll workers deviating from protocol, such as using the wrong type of PEB to open/close machines or using a Master PEB to cast votes.
- Calibration Error Detection: Flagging machines where votes were cast while the display might have been uncalibrated, potentially leading to unintended vote selections.
- Audit Data Completeness Checks: Verifying that all necessary audit data (e.g., ballot images) is present and consistent with other log entries.
AuditBear is designed to be publicly accessible and does not store any uploaded log data, ensuring voter privacy.
Datasets
The primary dataset used for this research is the South Carolina 2010 General Election audit logs. These logs were obtained from the "South Carolina Voting Information" website. The authors specifically focused on data from ES&S iVotronic DRE machines, which were widely deployed in that election. The paper also mentions considering other DRE systems like Sequoia AVC Edge and Hart eSlate but found their log export capabilities or data formats less suitable for their analysis.
Key Insights and Ideas
- Audit logs are a goldmine, but require specialized tools: The paper strongly emphasizes that despite their complexity, audit logs contain a wealth of information that can be automatically extracted to reveal critical election issues.
- Automated analysis is key to accessibility: Manual log analysis is impractical for election officials. An automated tool like AuditBear democratizes access to this data and its insights.
- Procedural errors are common and can be patterned: The research highlights that deviations from standard operating procedures by poll workers are not always random; they can indicate systemic training deficiencies.
- Time stamp accuracy is foundational: Incorrect machine clocks can significantly complicate and even invalidate audit log analysis, underscoring the need for accurate timekeeping.
- Voter privacy can be maintained: The authors demonstrate that analyzing audit logs for system and procedural issues does not inherently compromise voter privacy, as the logs themselves lack personally identifiable information.
Key Results
The analysis of the South Carolina 2010 election yielded several significant findings:
- Date/Time Errors: 1465 out of 4994 machines (approximately 29%) across 12 counties showed date adjustments during election day, suggesting potential issues like incorrect Daylight Savings Time algorithms.
- Missing Votes: 15 PEBs containing 2082 votes were not uploaded from 14 audited counties. Additionally, three machines were identified as not being closed, meaning their vote data was not uploaded.
- Long Lines: 671 out of 942 precincts (approximately 71%) stayed open late. The analysis identified specific time periods within these precincts where long lines were likely to have occurred.
- Procedural Errors: The study found instances of machines being opened and closed with different PEBs in 11 counties, with some precincts showing repeated occurrences, suggesting training needs.
- Calibration Errors: Seven counties had machines where votes were cast while the screen may have been uncalibrated.
- Incomplete Audit Data: Six counties showed inconsistencies between event logs and ballot image files, with Florence County having the most significant discrepancies.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Confirmation Challenges: For issues like long lines or machine malfunctions, there are limited resources for direct confirmation after the election.
- Tampering Assumption: The tool assumes audit logs are not maliciously tampered with; detecting such tampering is outside the scope.
- Data Completeness: The analysis relies on the completeness of the FOIA request for log data; errors in the request itself cannot be detected.
- Long Lines Analysis Scope: The current long lines analysis is primarily performed on precincts that stayed open late. Extending this to all precincts would require more sophisticated methods to account for variations in voter behavior across different locations.
- Future System Suggestions:
- Centralized Log Collection: Voting systems should support automatic generation and collection of audit logs in a central location.
- Universal Electronic File Format: Standardizing audit log formats would make tools like AuditBear more universally applicable.
- Event Meaning Documentation: Vendors should clearly document the meaning of all events in audit logs.
- Improved Time Logging: Enhancing the accuracy of date and time logging is crucial for many analyses.
- Public Availability of Manuals: Making voting system manuals publicly available would aid transparency.
- Capture Ballot Activation Events: Recording ballot activation times (not just cast votes) would improve long line analysis.
- Stronger Audit Log Standards: Future standards should mandate specific formats, electronic transmission, and voter privacy protections.
Summary updated