# Setup for new ProgramYear

As program requirements change from year to year, measure data and its schema are tracked separately per program year. To deal with this complexity, most scripts and all source data are also implemented on a per-year basis as to avoid the complexity of managing different schemas within a single script. As such, to set up a new program year, a number of new files/folders need to be created, copied, or modified. This is usually done on a long-running branch that reflects the incremental work done until the initial publishing required for benchmark generation.

- `/measures/`
  - A new folder for the new performance year needs to be created
  - a copy of the previous year's `measures-schema.yaml` should be placed in the folder, and any schema changes applied there.
- `/scripts/measures/`
  - a new folder for the new performance year should be created, and all the scripts from the previous PY should be copied to it. These scripts process input files in the `/util/measures/` folder that are sourced from 3rd parties, and might require updates from year to year.
  - a new `build-<performanceYear>-measures` script should be created, based on previous year's script, with the `currentPerformanceYear` variable set to the target performance year
  - update the `build-measures` 's `maxPerformanceYear` to the target performance year
  - files in `/measures/lib` are sources from other teams on the project and their format generally does not change from year to year, thus should not require updates
- `/staging/`
  - A new folder should be created for the new program year. It will be populated with intermediate files generated by the scripts.
- `/util/measures`
  - a new folder for the new program year should be created.
  - Files from the previous year can be copied over, preserving only the headers to serve as data stubs preventing the need to comment out their processing steps in the script files. `quality-strata` and `qcdr-measures-strata-names` should be copied in whole and updated as appropriate.

With these folders/files in place, you should be able to run `npm run build:measures <new performance year>` to generate a new `measures/<performanceYear>/measures-data.json` file that contains data sourced from the new `/util/measures` folder (an empty array in the case of only stub source files).

_________

## Notes on individual source files

### quality-measures

- Sourced from the cqm data warehouse xls. Script requires headers to be added to the data and the graphical headers removed after the spreadsheet is exported to a csv.
- The file is also missing program membership information, thus columns that populate `allowedPrograms` and `requiredByProgram` need to be added.
- strata are now included with the file, though the processing is still manual. Generally the strata do not change much from year to year, but sometimes updated descriptions need to be copied over to the `quality-strata.csv` as well as new entries created there for multiPerformanceRate measures.

### qcdr-measures

- Script works by column index. Thus if any new columns are added/removed the index mapping in the script needs to be updated.
- Strata descriptions are parsed from a column in the source file, however the names/ids for these strata are sourced from the `qcdr-measures-strata-names` file. These are matched to the strata based on measureId and stratum index, thus there need to be the right number of names as strata. Any new multiPerformanceRate measures should get new entries.

### pi-measures

- script works based off of headers, which are generally included in the file as a hidden row. Other headers/non data rows should be removed.
- any removed data (denoted as strike-through) needs to be manually removed before export to csv.