/** * @copyright Sister Software * @license AGPL-3.0 * @author Teffen Ellis, et al. * * `cordis`: the organization files CORDIS publishes for the participants in the EU's research * framework programmes, read for the address each participating organization registered. * * Input is the publisher's own CSV archives, one per programme, as * `https://cordis.europa.eu/data/cordis-projects-csv.zip` serves them for FP7 * (`fp7`), Horizon 2020 (`h2020`) and Horizon Europe (`HORIZON`). * Each archive holds an `organization.csv`: `;`-separated, every field double-quoted, one row per participation. * An organizationtherefore appears once for every project it took part in. * The adapter streams the member out of the archive through `readZipEntry` and `CSVSpliterator`, * so no extracted copy sits on disk. * `opts.inputPath` is one archive, one extracted `organization.csv`, or a directory of archives. * `fetchCORDIS` writes the directory form. * * Measured on the archives fetched 2026-10-03, the three members hold 420,811 participation rows and * 78,842 distinct published records, and the adapter writes 57,930 rows over 158 countries. * * ## The columns read * * `name`, `shortName`, `street`, `postCode`, `city` and `country` reach a row or decide whether * one is written. `organisationID`, `activityType`, `SME`, `vatNumber` and `nutsCode` were read while * deciding the rules below and reach no row. * * ## Organizations, and no natural person * * A corpus holds the addresses of organizations and no address of an identifiable natural person. * CORDIS publishes no legal-form column. * `activityType` takes the five values `HES`, `PRC`, `REC`, `OTH` and `PUB`, plus an empty value on * older rows, and no value marks a natural person. * A natural person still enters the participant register: a sole trader, or a researcher contracted * in their own name, is registered under their own name, mostly with `activityType` `PRC`. * Measured over the three archives, the names of those registrations take three forms, and * {@linkcode naturalPersonForm} refuses each: * * 1. **Every word is a personal name.** `FISK ADRIAN`, `SCHWARZ KAI-UWE WOLFGANG`, `Kerstin * Minnich`: two to four words, each a given name or surname in libpostal's `all` dictionaries, * at least one a given name. 194 records. * 2. **The participant register's `SURNAME Given` casing.** `BUCKENHUSKES Herbert Johannes`: an * upper-case run followed by title-case words that are all personal names, one of them a given * name, for a surname the dictionary lacks. 5 records. * 3. **A sole-trader legal form.** German `e.K.`, `e.Kfm.` and `e.Kfr.`, and `Inh.` (owner), in the * name or the short name, as in `Melina Bucher e.Kfr.`. 3 records. * * Neither name rule fires on a name holding a word that is a country's name, so `ERICSSON FRANCE` and * `IDEMIA Public Security France` are admitted. * * The check is a refusal rule rather than a classifier. A sole trader trading under a business name * that holds no personal name passes it. Some organizations are refused: `JOHN COCKERILL` by the first * rule, and `UAB Nando` and `US Forest Service` among the 5 the casing rule refuses. The count each * rule refuses is in the `dropped` map. The natural person a reader would expect first, * `CORDOVA Trey Christian`, publishes no street, postcode or city at all and is refused as * {@linkcode CORDISRefusal.LocalityAbsent}. * * The publisher's legal notice states the same limit from its side: reuse may require clearing * additional rights "if a specific content depicts identifiable private individuals". * * ## Country codes * * CORDIS writes EU country codes, and those differ from ISO 3166-1 in two places: `EL` is Greece and * `UK` is the United Kingdom. Both are mapped, to `GR` and `GB`. Any other value that is not an ISO * alpha-2 code refuses the row: `XK` (Kosovo, a user-assigned code, 34 records), `ZZ` (1), an empty * value (323), and `DE;HU` (1), two countries in one field. * * A country whose codex `layoutForCountry` is `null` has no rendering to write the row in. * The row is refused with {@linkcode CORDISRefusal.CountryNoLayout} rather than rendered in another * country's order: 353 records over 37 countries, Ghana's 77 the most. * * ## The street line * * `street` holds the whole street line as the participant typed it, usually but not always in the * country's own order: `THE WELLCOME TRUST LIMITED` writes `EUSTON ROAD 215 GIBBS BUILDING`, a * number-last line in a number-first country. The house number is therefore split off only on the * side the country's order puts it, and {@linkcode streetOrderForCountry} reads that side from codex's * `house-number-precedes-street` claim. A country whose observations disagree, or state no side, * has no unambiguous side, and its lines are kept only where they hold no digit. * * **A street line holding a digit the split could not place refuses the row**, as `it-anac` does, and * so does a line whose remainder still holds a digit once the number is split off. `C GRAN DE GRACIA * NUM 1, PLANTA 4, PUERTA 3` would otherwise label `3`, the door, as the house number, and `TRIERSTRAAT * 49 STRATENPLAN BUS 7` would label the box. The cost is the street whose name holds a digit, `PLACE DU * 20 AOUT 7`, which is refused with the rest. These two rules refuse 14,438 and 4,646 records, * the largest share of the 20,911 refused, led by France, Great Britain and Spain. * * A civic-number marker between the street and the number (`NR`, `NO`, `NUM`, `NÂș`) is dropped, so the * street is `HAUPTSTRASSE` rather than `HAUPTSTRASSE NR.`. A kilometer point, `CTRA SANT LLORENC DE * MORUNYS KM, 2`, is refused rather than read as a house number. * * ## Placeholders and the postcode * * A value holding no letter and no digit (`-`, `.`), or one of `N/A`, `NA`, `none`, `Not applicable`, * is treated as absent. A postcode with no digit (`EIRE`, `Clare`, `POSTFACH`) and a postcode of * zeroes alone are dropped from the row rather than refusing it, and counted as `component:postcode:*`. * A postcode written with its country prefix, `LV-1063` or `LT-01103`, is kept as published: Latvia * and Lithuania write that prefix in their official form. * * A layout with no slot for a published component leaves it out, and the row is still written: * the United Arab Emirates' layout has no locality line, and Japan's Latin layout none below the * prefecture. Each is counted as `component::unplaced`. * * ## Deduplication and identity * * An organizationappears once per project, across three programmes. The adapter reads each distinct * published record once, keyed on the five fields that reach a row, and writes each distinct rendered * address once. A record whose address changed between programmes contributes one row per version. * The row id is content-addressed over the aligned components. * * ## License * * The CORDIS legal notice at `https://cordis.europa.eu/about/legal`, read 2026-10-03, states: * "Unless otherwise noted (e.g. in individual copyright notices), the reuse of the editorial content on * this website owned by the EU is authorized under the Creative Commons Attribution 4.0 International * (CC BY 4.0) licence." The same notice records that "The Commission's reuse policy is implemented by * Commission Decision 2011/833/EU of 12 December 2011 on the reuse of Commission documents", and the * data.europa.eu records for `cordisfp7projects`, `cordish2020projects` and * `cordis-eu-research-projects-under-horizon-europe-2021-2027` label each CSV distribution with that * decision (`COM_REUSE`). CC BY requires credit and a statement that changes were made, so every row * records the license and {@linkcode CORDIS_ATTRIBUTION} is the credit the model card prints. * * The adapter honors `opts.limit`, `opts.signal` and `opts.country`. */ import { type PathBuilderLike } from "path-ts"; import { type SplitStreetLine } from "#adapters/street-line"; import { type CanonicalRow, type CorpusAdapter } from "#types"; /** * Registry id for this adapter. * * Stamped into every row it emits, so a corpus record can be traced back to the dataset it came from. */ export declare const CORDIS_ADAPTER_ID = "cordis"; /** * The license the CORDIS legal notice states for content the EU owns. */ export declare const CORDIS_LICENSE = "CC-BY-4.0"; /** * The credit CC BY 4.0 obliges. * * The model card prints it beside the license and a statement that the data was changed. */ export declare const CORDIS_ATTRIBUTION = "\u00A9 European Union, CORDIS (Publications Office of the European Union)"; /** * The archive member that holds the participants in every programme's CSV zip. */ export declare const CORDIS_ORGANIZATION_MEMBER = "organization.csv"; /** * The columns the adapter indexes by the publisher's own spelling. * * Checked against the first record, so a renamed column raises instead of * reaching every row as an empty string. */ export declare const CORDIS_REQUIRED_COLUMNS: readonly string[]; /** * The two places CORDIS's EU country codes differ from ISO 3166-1 alpha-2. */ export declare const CORDIS_COUNTRY_ALIASES: Readonly>; /** * Why a published record did not become a row. */ export declare const CORDISRefusal: { /** * `country` is not an ISO 3166-1 alpha-2 code once `EL` and `UK` are mapped. */ readonly CountryNotISO: "country-not-iso"; /** * Codex has no layout for the country, so no rendering states where each component goes. */ readonly CountryNoLayout: "country-no-layout"; /** * Every word of the name is a personal name. */ readonly NamePersonalWords: "name-personal-words"; /** * The name takes the participant register's `SURNAME Given` casing. */ readonly NamePersonalCasing: "name-personal-casing"; /** * The name or short name holds a sole-trader legal form. */ readonly NameSoleTrader: "name-sole-trader"; /** * The record states no city. */ readonly LocalityAbsent: "locality-absent"; /** * The street line holds a digit and no house number could be split off on the country's side. */ readonly StreetNumberUnplaced: "street-number-unplaced"; /** * A house number was split off, and the street that remains still holds a digit. */ readonly StreetDigitInRemainder: "street-digit-in-remainder"; /** * Beside the locality, the record holds neither a street nor a postcode. */ readonly ComponentsTooFew: "components-too-few"; /** * The components do not render in the country's layout. */ readonly Unrenderable: "unrenderable"; }; /** * One of the {@link CORDISRefusal} reasons. */ export type CORDISRefusal = (typeof CORDISRefusal)[keyof typeof CORDISRefusal]; /** * What one published record yielded. * * `discarded` lists the `component::` keys of values kept out of an admitted row. */ export type CORDISReading = { readonly admitted: CanonicalRow; readonly discarded: readonly string[]; } | { readonly refused: CORDISRefusal; readonly country: string | null; }; /** * The columns of one `organization.csv` record the adapter reads. */ export interface CORDISOrganizationRecord { organisationID?: string; name?: string; shortName?: string; activityType?: string; SME?: string; vatNumber?: string; street?: string; postCode?: string; city?: string; country?: string; nutsCode?: string; } /** * The given names and surnames the personal-name rules consult, lower-cased. */ export interface PersonNameLexicon { readonly givenNames: ReadonlySet; readonly surnames: ReadonlySet; } /** * Load libpostal's language-independent given-name and surname dictionaries. */ export declare function loadPersonNameLexicon(): Promise; /** * Which side of the street name a country writes the house number on. */ export declare const StreetOrder: { readonly NumberFirst: "number-first"; readonly NumberLast: "number-last"; }; /** * One of the {@link StreetOrder} values. */ export type StreetOrder = (typeof StreetOrder)[keyof typeof StreetOrder]; /** * The side a country writes its house number on, or `null` where codex's sources disagree or state none. * * Read from the `house-number-precedes-street` claim, which collects libaddressinput's `fmt` and `lfmt`, * the OpenCage street order, the board's hand-authored layout and a retrieved UPU S42 template. * A side is returned only when at least one source states it and none states the other. * * Hong Kong's Chinese-script `fmt` writes the number after the street * and its Latin `lfmt` before it, so Hong Kong has no side. */ export declare function streetOrderForCountry(country: string): StreetOrder | null; /** * Whether a published value is a placeholder for an absent one, or holds no letter and no digit. */ export declare function isCORDISPlaceholder(value: string): boolean; /** * The personal-name form a participant's name takes, or `null` where it takes none. * * Exported so a test and a measurement can read the rule the adapter applies rather than a copy of it. */ export declare function naturalPersonForm(name: string, shortName: string, lexicon: PersonNameLexicon): CORDISRefusal | null; /** * The street line split on the side the country writes its number, or the refusal the line meets. * * `order` is `null` for a country with no unambiguous side, and such a line * is kept only where it holds no digit. */ export declare function splitCORDISStreetLine(line: string, order: StreetOrder | null): SplitStreetLine | { refused: CORDISRefusal; } | null; /** * The ISO 3166-1 alpha-2 code for a CORDIS `country` value, or `null` for one that maps to none. */ export declare function cordisCountryCode(value: string | null | undefined): string | null; /** * Read one published record into a row, or report why it yielded none. * * Exported because the refusal reasons are the measurement this adapter is checked by. */ export declare function readCORDISRecord(record: CORDISOrganizationRecord, lexicon: PersonNameLexicon): CORDISReading; /** * One distinct published record and what it yielded. */ export interface CORDISRecordReading { readonly record: CORDISOrganizationRecord; readonly reading: CORDISReading; } /** * Every distinct published record under `inputPath`, read once each. * * A record is distinct on the five fields that reach a row, so an organization that took part in many * projects with one address is read once, and one whose address changed is read once per version. * `onDuplicate` is called once for every participation row a record already read covers. * * The adapter and the measurement both read through this, so the counts a * measurement reports are the adapter's own. */ export declare function readCORDISRecords(inputPath: PathBuilderLike, lexicon: PersonNameLexicon, options?: { signal?: AbortSignal; onDuplicate?: () => void; }): AsyncGenerator; export declare function createCORDISAdapter(): CorpusAdapter; /** * The configured adapter instance registered with the corpus builder. */ export declare const cordisAdapter: CorpusAdapter; //# sourceMappingURL=adapter.d.ts.map