SF Seal

SF Open Source Voting TAC

Official site of the San Francisco Open Source Voting System Technical Advisory Committee (OSVTAC)


Open Source Voting System Project Recommendations

(Approved by OSVTAC on March 8, 2018.)

Last posted: March 19, 2018

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. For copyright and attribution information for this work, see this section. The source files for the text can be found on GitHub here.

6. Recommendations

6.1. Interim Voting System

6.2. Requirements-gathering

This section contains recommendations related to gathering requirements. For committee recommendations of specific requirements, see the Requirements section below.

[TODO]

6.3. Requirements

This section lists some of the requirements the system should satisfy.

6.3.1. Accessibility

6.3.2. Other

6.4. Project Management

6.5. Open Source

This section covers topics related to open source.

6.6. Procurement

[TODO]

6.7. Software architecture and design

6.8. Software development

6.9. Hardware design

[TODO]

6.10. Documentation

[TODO]

6.11. Security

[TODO]

6.12. Testing

  1. Gather real election data. Datasets of real election data (e.g. a couple past elections in San Francisco of different types) should be compiled in a structured format for product prototyping and testing. This includes not just vote totals but also candidate and contest data. This will help in establishing requirements and designing the system.

  2. Gather real digital ballot pictures. Starting with the June 2018 election, during each election the Department should gather and save large numbers (e.g. thousands) of digital ballot pictures for future testing purposes. The Director has already expressed a willingness to do this in the case that the voting system supports it. The Department should do this during the canvass after each election because it may not be possible to obtain ballot pictures after the ballots are physically sealed and eventually destroyed. Having a variety of real-world digital ballot pictures will aid in developing and testing the ballot picture interpreter component, even if the ballot design is different from what will eventually be used. Also, using real ballots can provide test cases that might not be thought of if trying to construct test cases manually.

    [Item added: Dec. 14, 2017 meeting.]

  3. Stand-alone test data. In the course of developing the open source voting system, where possible, structure and store test data separate from the software application (e.g. in separate repositories) and in an application-agnostic form (e.g. using open data formats). These can be separate deliverables. The test data should include both test inputs and, when appropriate, test outputs (aka test expectations). Doing this allows the test data to be used by other applications and in particular could help facilitate additional open source implementations of components. Making the test data independent and more easily available can also improve the quality and correctness of the test data, for example by making it easier for others to check or add more test cases.

    This recommendation makes more sense for higher level end-to-end tests rather than lower-level tests like unit tests since unit tests are often tied to a particular implementation. Examples of test cases for higher-level tests include things like (1) for the ballot picture interpreter component, a digital ballot picture as the input and the corresponding cast vote record as the output, and (2) for the RCV tabulator, the cast vote records for an RCV contest as the input and the round-by-round vote totals as the output.

    [Item added: Dec. 14, 2017 meeting.]

6.13. Certification

[TODO]

6.14. Hardware manufacturing or assembly

[TODO]

6.15. Deployment

[TODO]

6.16. Software maintenance

[TODO]

6.17. Hardware maintenance

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. For copyright and attribution information for this work, see this section. The source files for the text can be found on GitHub here.