| Dataset | Language | Source | |-------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| | Parlamint corpus | at, bg, cz, dk, ee, es, es-ga, fi, fr, gb, gr, hr, hu, it, lv, nl, no, pl, pt, rs, se, si | (Erjavec et al., 2021) | | Crawl of Bulgarian news websites | bg | [Link](http://old.dcl.bas.bg/dataset/Bulgarian_news.7z) | | Bulgarian National Corpus | bg | [Link](http://old.dcl.bas.bg/dataset/BulNC.7z) | | Colossal OSCAR 1.0 | bg, ca, cs, cy, da, de, el, en, es, et, eu, fi, fr, ga, gl, hr, hu, it, lt, lv, mt, nl, nn, no, oc, pl, pt, ro, ru, sh, sk, sl, sr, sv, uk | (Brack et al., 2024) | | Wikimedia dumps | bg, ca, cs, da, de, el, en, es, et, eu, fi, fr, ga, gl, hr, hu, it, lt, lv, mt, nl, nn, no, pl, pt, ro, sh, sk, sl, sr, uk | [Link](https://dumps.wikimedia.org/) | | OpenSubtitlesv2016 | bg, ca, cs, da, de, el, en, es, et, eu, fi, fr, gl, hr, it, lt, lv, nl, no, pl, pt, ro, sk, sl, sr, sv, uk | (Lison & Tiedemann, 2016) | | MaCoCu web corpus | bg, ca, el, hr, mt, sl, sr, uk | (Bañón et al., 2022) | | EurLEX-Resources | bg, cs, da, de, el, en, es, et, fi, fr, ga, hr, hu, it, lt, lv, mt, nl, pl, pt, ro, sk, sl, sv | [Link](https://huggingface.co/datasets/joelniklaus/eurlex_resources) | | MC4-Legal | bg, cs, da, de, el, en, es, et, fi, fr, ga, hu, it, lt, lv, mt, nl, pl, pt, ro, sk, sl, sv | [Link](https://huggingface.co/datasets/joelito/legal-mc4) | | CURLICAT Corpus | bg, hr, hu, pl, ro, sk, sl | (Váradi et al., 2022) | | CATalog | ca | (Palomar-Giner et al., 2024) | | Spanish Crawling | ca, es, eu, gl | Relevant Spanish websites crawling | | Starcoder | code | (Li et al., 2023) | | SYN v9: large corpus of written Czech | cs | (Křen et al., 2021) | | Welsh-GOV | cy | Crawling from [Link](https://www.llyw.cymru) | | DaNewsroom | da | (Varab & Schluter, 2020) | | The Danish Parliament Corpus 2009 - 2017, v1 | da | (Hansen, 2018) | | Danish GigaWord | da | (Strømberg-Derczynski et al., 2021) | | DK-CLARIN Reference Corpus of General Danish | da | [Link](https://korpus.dsl.dk/clarin/) | | Open Legal Data - German court decisions and laws | de | (Ostendorff et al., 2020) | | DeWaC | de | [Link](https://docs.sslmit.unibo.it/doku.php?id=corpora:dewac) | | Greek Web Corpus | el | (Outsios et al., 2018) | | Greek Legal Code | el | (Papaloukas et al., 2021) | | BIGPATENT | en | (Sharma et al., 2019) | | peS2o | en | (Soldaini & Lo, 2023) | | PG-19 | en | (Rae et al., 2019) | | proof-pile | en | [Link](https://huggingface.co/datasets/hoskinson-center/proof-pile) | | Auxiliary Mathematics Problems and Solutions (AMPS) dataset | en | (Hendrycks et al., 2021) | | Pile of Law (selected subsets) | en | (Henderson\* et al., 2022) | | RedPajama-Data T1 (StackExchange subset) | en | (Computer, 2023) | | The Pile (PhilPapers subset) | en | (Gao et al., 2021) | | Spanish Legal Domain Corpora | es | (Gutiérrez-Fandiño et al., 2021) | | HPLTDatasets v1 - Spanish | es | (de Gibert et al., 2024) | | Legal | es | Internally generated legal dataset: BOE, BORME, Senado, Congreso, Spanish court orders, DOGC | | Biomedical | es | Internally generated scientific dataset: Dialnet, Scielo, CSIC, TDX, BSC, UCM | | Scientific | es | Internally generated scientific dataset: Wikipedia LS, Pubmed, MeSpEn, patents, clinical cases, medical crawler | | Estonian National Corpus 2021 | et | (Koppel & Kallas, 2022) | | Estonian Reference Corpus | et | [Link](https://www.cl.ut.ee/korpused/segakorpus/) | | EusCrawl (filtered: no Wikipedia, no NC-licenses) | eu | (Artetxe et al., 2022) | | Latxa Corpus v1.1 | eu | (Etxaniz et al., 2024) [Link](https://huggingface.co/datasets/HiTZ/latxa-corpus-v1.1)| | Yle Finnish News Archive | fi | [Link](http://urn.fi/urn:nbn:fi:lb-2021050401) | | CaBeRnet: a New French Balanced Reference Corpus | fr | (Popa-Fabre et al., 2020) | | French Public Domain Newspapers | fr | [Link](https://huggingface.co/datasets/PleIAs/French-PD-Newspapers) | | French Public Domain Books | fr | [Link](https://huggingface.co/datasets/PleIAs/French-PD-Books) | | The Gaois bilingual corpus of English-Irish legislation (Irish legislation) | ga | [Link](https://portulanclarin.net/repository/browse/the-gaois-bilingual-corpus-of-english-irish-legislation-processed/daeac17c9e3511ea9b7f02420a000407b83de243dc0b469aab41084386c5b80f/) | | Irish Universal Dependencies | ga | [Link](https://universaldependencies.org/ga/index.html) | | CorpusNÓS | gl | (de-Dios-Flores et al., 2024) | | Croatian web corpus hrWaC 2.1 | hr | (Ljubešić & Klubička, 2014) | | ITWaC | it | [Link](https://docs.sslmit.unibo.it/doku.php?id=corpora:itwac) | | Corpus of State-related content from the Latvian Web (Processed) | lv | [Link](https://catalog.elra.info/en-us/repository/browse/ELRA-W0169/) | | Korpus Malti | mt | (Micallef et al., 2022) | | SoNaR Corpus NC 1.2 | nl | [Link](https://taalmaterialen.ivdnt.org/download/tstc-sonar-corpus/) | | Norwegian Colossal Corpus | nn, no | (Kummervold et al., 2021) | | Occitan Corpus | oc | Provided by [IEA](https://www.institutestudisaranesi.cat/) | | Polish Parliamentary Corpus / Korpus Dyskursu Parlamentarnego | pl | (Ogrodniczuk, 2018) | | NKJP-PodkorpusMilionowy-1.2 (National Corpus of Polish) | pl | (Lewandowska-Tomaszczyk et al., 2013) | | Brazilian Portuguese Web as Corpus | pt | (Wagner Filho et al., 2018) | | ParlamentoPT | pt | (Rodrigues et al., 2023) | | MARCELL Romanian legislative subcorpus v2 | ro | [Link](https://elrc-share.eu/reposMARCELL%20Romanian%20legislative%20subcorpus%20v2itory/browse/marcell-romanian-legislative-subcorpus-v2/2da548428b9d11eb9c1a00155d026706ce94a6b59ffc4b0e9fb5cd9cebe6889e/) | | Korpus slovenských právnych predpisov v1.9 | sk | [Link](https://www.juls.savba.sk/data/marcell/legal-sk-20220322-1.9.ver.xz) | | od-justice 2.0 | sk | [Link](https://www.juls.savba.sk/data/od-justice/od-justice-2.0.ver.xz) | | Corpus of academic Slovene KAS 2.0 | sl | (Žagar et al., 2022) | | slWaC web corpus | sl | (Erjavec et al., 2015) | | SrpKorSubset (news, legal, academic, conversation, literary) | sr | [Link](http://www.korpus.matf.bg.ac.rs/) | | The Swedish Culturomics Gigaword Corpus | sv | (Rødven-Eide, 2016) | | Corpus of laws and legal acts of Ukraine | uk | [Link](https://lang.org.ua/en/corpora/#anchor7) |