Under the direction of CTO-EDS&AI, the Contractor shall design, build, adapt, execute and maintain data processing
pipelines within the NCIA classified sandbox environment.

• Setting up / improving pipelines to process all required documents and that uniquely identifies and traces
  decisions and processing steps. This is to be conducted on the provided classified sandbox environment, with
  provided performance hardware and toolsets;
• Implementing / improving (missing) pipeline steps for marking duplicate files, based on file attributes, path
  (structure) and content (similarity), and rules for considering a file or structure a duplicate;
• Extracting document-format records from Functional Area Systems (FAS) databases and back-ups performed
  otherwise. Archiving SME’s and system SME’s are available for guidance on target formats and source system
  structure and data interpretation. Each FAS is processed separately;
• Processing / Monitoring progress of various office, image and video file types to the accepted archiving
  formats, including extraction of metadata and preparing search semantic indexes;
• Automating registering all processed documents with semantic indexes with the sandbox natural language
  search tool;
• Automating the final copy of all non-duplicate and extracted archive documents with content and metadata
  to the NATO archiving system;
• Reporting status, progress and statistics of the (raw) files being processed to archive formats, metadata and
  search indexes;
• Delivering full reporting of results, trace of pipeline steps taken and (stakeholder) accepted failures.
  Quarterly updates.