“The Department of Commerce (DOC), National Oceanic and Atmospheric Administration (NOAA), National Marine Fisheries Service (NMFS), Northeast Fisheries Science Center (NEFSC) is seeking a contractor to develop software solutions that closely align the image review process currently underway as part of the Species Verification Program (SVP) with machine learning/deep neural network based analysis. Specifically, we seek a contractor to develop software with detailed documentation that applies machine learning (ML) algorithms to identify 45 common fish species and incorporate continuous learning to improve models as new images and verification data become available. The goal of this work is to produce software that can maintain or improve accuracy and increase efficiency of the SVP data processing and be shared with others working to identify similar species. The Contractor personnel shall be working under the guidance of NMFS NEFSC staff.
The base period of performance will be twelve (12) months and will include project scoping, algorithm development, testing and evaluation, production of ML algorithms, development of a database or software platform to incorporate results into SVP data acquisition and notation, and the creation of accompanying documentation. The project is expected to begin in late spring of 2020 with meetings to coordinate the planning and specification of technical requirements. Details of the software including specific algorithms and output, where and how the algorithms and output will be stored and accessed (e.g., via cloud based or in house NOAAGPU computers), specific requirements for integration with and/or development of a front-end user interface will be finalized in the initial scoping period. The scoping period shall entail the development of a project plan as well as a project timeline and governance structure. The Contractor shall be responsible for developing algorithms for identifying species including potentially annotating images with bounding boxes for locating the target specimens. In addition, the Contractor shall create a database platform that would, at minimum, load output into tables matching existing templates (e.g., output converted from soft-max layer probabilities) to facilitate integration with ongoing SVP data processing (Oracle/SQL or compatible). The algorithm(s) shall allow for continuous model improvement/retraining by automatically incorporating new SVP verified images and data. In addition, the Contractor shall submit a draft report for comments and then final report describing the models’ development, developed code, detailed directions for use, and a summary of top-1 accuracy by species based upon 20% of the data withheld from algorithm testing species.
The Contractor will be able to access NOAA FSB’s Species Verification library of images to develop models, produce and test the deep learning algorithm. The development of software solutions that allow application of machine learning models to existing and additional accumulating images and data, and the creation of a database or platform to incorporate output into the SVP data stream shall be primarily the responsibility of the Contractor. However, NMFS will assist and provide feedback on development and comments on incorporating results at regular intervals. Algorithms, data processing code (scripts including those to produce image annotations), and software for this project are encouraged to be made available as open source code with associated documentation published on GitHub. However, images and data are to remain the exclusive property of the U.S. Government. We also require the Contractor to maintain images, ML models and code, and associated software for a period of up to 3 years from the contract start date, but all supplied photos and data must be returned to the NEFSC Fisheries Sampling Branch (FSB) prior to the end of this period. Our intention is to make the algorithms, associated software, and documentation available to the public, and so the Contractor is encouraged to produce these in a form that is open source where that can be shared in a public domain, but certain privacy issues still require resolution prior to any publication. The Contractor shall work with NMFS staff to develop software solutions that can be of use to the broader scientific community.
This development aligns with NMFS’s longer-term goal of implementing machine learning in fisheries monitoring to aid in the development of resources with which to enhance the efficiency and accuracy of automated species identification.”