Dataset

DBnary provides lexical data extracted from 28 Wiktionary language editions, modeled with the W3C Ontolex-Lemon vocabulary. Each language edition is re-extracted as soon as a new Wiktionary dump is published (once a month since 2024, more frequently before). As it is derived from Wiktionary, the data is distributed under Creative Commons Attribution-ShareAlike 4.0.

The dataset is registered on the datahub. It may be downloaded (see below) or queried online.

Languages

Linguistic data is extracted from the Bulgarian, Catalan, Chinese, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Indonesian, Irish, Italian, Japanese, Kurdish, Latin, Lithuanian, Malagasy, Norwegian, Polish, Portuguese, Russian, Serbo-Croatian, Spanish, Swedish and Turkish language editions.

For each language edition l, DBnary provides two lexicons:

  • the endolexicon: data on language l, as described in the l language edition (e.g. French words in the French Wiktionary);
  • the exolexicon (since January 2022): data on all other languages described in the l language edition (e.g. English or Japanese words in the French Wiktionary). Altogether, exolexica cover thousands of languages.

Download

Latest extracts are available in the latest folder, in Turtle format (bzip2 compressed). For each language edition (xx is its 2-letter code), the data is split in several files:

FileContent
xx_dbnary_ontolexThe core endolexicon, as extracted from Wiktionary: pages, lexical entries, senses, translations, lexical relations…
xx_dbnary_exolex_ontolexThe core exolexicon, same content for all other languages described in the xx edition.
xx_dbnary_lime, xx_dbnary_exolex_limeMetadata of the endo and exo lexicons (version, number of elements), modeled with the Ontolex LIME module.
xx_dbnary_enhancementDisambiguated translations: links from each translation to the sense(s) of the source entry for which it is valid. They are computed with a heuristic (presented at LREC 2014, see publications) with state of the art accuracy, in sync with the core data.
xx_dbnary_morphologyExtensive morphology: the inflected forms of lexical entries, with their linguistic annotations. Not available for all languages.
xx_dbnary_etymologyEtymological data (currently English edition only).
xx_dbnary_statisticsStatistics on the extracted data, modeled with the Data Cube vocabulary (see Statistics and quality).

Previous versions

Each language folder (e.g. /static/ontolex/fr) keeps the most recent previous versions. Versions modeled with lemon (before July 2017) and older ontolex versions are archived on Zenodo DOI.

Data format

DBnary uses an extended version of Ontolex. The DBnary vocabulary is documented here.

URIs and graphs

Every resource is identified by a dereferenceable URI under http://kaiko.getalp.org/dbnary/<lang>/, where <lang> is the ISO 639-3 code of the language edition (eng, fra, …). For instance:

Online, each lexicon is a named graph: /dbnary/eng/ for the English endolexicon, /dbnary/eng_exolex/ for the English exolexicon.

Model

flowchart LR
  P[dbnary:Page] -- dbnary:describes --> E[ontolex:LexicalEntry]
  E -- ontolex:canonicalForm --> F[ontolex:Form]
  E -- ontolex:sense --> S[ontolex:LexicalSense]
  T[dbnary:Translation] -- dbnary:isTranslationOf --> E
  T -. "dbnary:isTranslationOf (enhancement)" .-> S
  P -- "dbnary:synonym, dbnary:hypernym…" --> P2[dbnary:Page]
  S -- "dbnary:synonym, dbnary:hypernym…" --> P2
ElementMain properties
dbnary:Pagedbnary:describes the lexical entries of a Wiktionary page.
ontolex:LexicalEntryontolex:canonicalForm (an ontolex:Form with its ontolex:writtenRep and pronunciations as ontolex:phoneticRep), lexinfo:partOfSpeech, lime:language, ontolex:sense.
ontolex:LexicalSensedbnary:senseNumber, as numbered in Wiktionary, skos:definition and skos:example (resources whose text is their rdf:value).
dbnary:Translationdbnary:isTranslationOf its source entry (and sense, in enhancement files), dbnary:targetLanguage (a lexvo URI, e.g. lexvo:deu), dbnary:writtenForm, dbnary:gloss (a dbnary:Gloss, the sense label given in Wiktionary’s translation table).
Lexical relationsdbnary:synonym, dbnary:antonym, dbnary:hypernym, … link a page, an entry or a sense to the target dbnary:Page, as Wiktionary does not tell which entry of the target page is meant.
Inflected forms (morphology files)ontolex:otherForm links an entry to each ontolex:Form with its ontolex:writtenRep, ontolex:phoneticRep and grammatical features. The features depend on the language edition, e.g. lexinfo:gender and lexinfo:number for French nouns and adjectives, olia:hasMood for German verbs.
lime:LexiconMetadata of each endo or exo lexicon.

The prefixes used in this page and in the online sample queries:

@prefix dbnary:     <http://kaiko.getalp.org/dbnary#> .
@prefix dbnary-eng: <http://kaiko.getalp.org/dbnary/eng/> .
@prefix ontolex:    <http://www.w3.org/ns/lemon/ontolex#> .
@prefix lexinfo:    <http://www.lexinfo.net/ontology/2.0/lexinfo#> .
@prefix lime:       <http://www.w3.org/ns/lemon/lime#> .
@prefix lexvo:      <http://lexvo.org/id/iso639-3/> .
@prefix skos:       <http://www.w3.org/2004/02/skos/core#> .
@prefix rdf:        <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix olia:       <http://purl.org/olia/olia.owl#> .

For instance, a simplified extract of the English page cat (forms, senses and translations have their own URIs in the data; they are shown here as blank nodes, except the sense the translation points to):

dbnary-eng:cat a dbnary:Page ;
    dbnary:describes dbnary-eng:cat__Noun__1 ;
    dbnary:hypernym dbnary-eng:feline .

dbnary-eng:cat__Noun__1 a ontolex:LexicalEntry ;
    lime:language "en" ;
    lexinfo:partOfSpeech lexinfo:noun ;
    ontolex:canonicalForm [ a ontolex:Form ;
        ontolex:writtenRep "cat"@en ;
        ontolex:phoneticRep "/ˈkæt/"@en-fonipa-fonipa ] ;
    ontolex:sense dbnary-eng:__ws_1.1.1_cat__Noun__1 .

dbnary-eng:__ws_1.1.1_cat__Noun__1 a ontolex:LexicalSense ;
    dbnary:senseNumber "1.1.1" ;
    skos:definition [ rdf:value "A carnivorous, four-legged, generally furry domesticated species (Felis catus) of feline animal, commonly kept as a house pet."@en ] .

[] a dbnary:Translation ;
    dbnary:isTranslationOf dbnary-eng:cat__Noun__1 ,
        dbnary-eng:__ws_1.1.1_cat__Noun__1 ;   # enhancement
    dbnary:gloss [ a dbnary:Gloss ; rdf:value "domestic species"@en ] ;
    dbnary:targetLanguage lexvo:deu ;
    dbnary:writtenForm "Katze"@de .

Statistics and quality

The Dashboard shows the number of pages, entries, senses, translations, lexical relations… available globally or in each language edition, and their evolution.

As the extraction goes on for years, Wiktionary communities sometimes change the way they represent lexical information, and the extractors get out of sync. A decrease in the number of elements usually reveals such a change; we then adapt the extractor to re-synchronize it with Wiktionary data. The whole history of statistics is available in RDF (Data Cube vocabulary) and may be queried along with DBnary data through the SPARQL endpoint.

A short history of DBnary

The first version of DBnary was released in August 2012 as a participation to the Monnet Challenge for Lexical Linked Data, with a few languages (mainly English, French, German, Italian and Portuguese). From the beginning, the extraction has been designed as an ongoing process where each Wiktionary dump is extracted as it is produced, so that the dataset follows the evolution of Wiktionary (and of languages). New language editions were added over time.

Until July 2017, the dataset was modeled with the lemon vocabulary; all extractors then switched to Ontolex, which extends lemon and is a W3C community specification.

How to cite

Sérasset Gilles (2015). DBnary: Wiktionary as a Lemon-Based Multilingual Lexical Resource in RDF. Semantic Web Journal, 6(4), pp. 355-361. doi:10.3233/SW-140147