Extraction

More examples extracted from Wiktionary

Wiktionary dictionary entries sometimes have examples or citation and we already extracted some of them in certain languages. With the work from Emmanuel Mayoux-Li, our undergrad intern, we added more languages and also fixed obsolete extractors to better get such examples and citations. For instance, this is more the 350,000 examples in German that were rencently added to the DBnary dataset. Such examples may be interesting for researchers that want illustrations of a word sense usage in a real sentence.

Read More

Examples are now extracted from the English version

The english extractor should now correctly extract examples and citations (along with references) from the English Language Edition. Such examples where initially extracted for the French Edition and we will try to extend such extraction for all other editions.

Read More

Turkish extractor has been rewritten

Since almost one year, the Turkish data was empty as the extractor was not fixed after the Turkish wiktionary community changed drastically the way the pages were encoded. This has now been fixed with a new version of the Turkish extractor that relies on the new mediawiki parser that is part of the DBnary extractor infrastructure.

Read More

Swedish Morphology Extraction is available

The DBnary extractor now extracts the Swedish morphology. This data is available from the 20200701 extraction version, that used the 2.3.1 version of the extractor. The morphological data may be downloaded and will be available as soon as we upload new datasets to the live virtuoso server. The extraction remain a little bit drafty, hence if you spot any mistakes or missing information (that is available in the morphology tables of the entry page), then fill free to create an issue on the DBnary extractor’s bitbucket forge.

Read More

We are handling Thesaurus entries in English dataset

The English extractor has recently been extended to handle Thesaurus entries. This leads to more lexico-semantic relations. However, such relations originates and leads to Pages rather than to LexicalEntries. This shortcoming will be handled during data enhancement (still to come). Happy Linking !

Read More

Major bug discovered and fixed

While using DBnary in conjunction with other lemon resources (mainly during the Lider datathon in Madrid), we discovered a small but major problem with DBnary data. Until now, the lemon prefix was http://www.lemon-model.net/ while the official prefix was http://lemon-model.net/ leading to a poor mapping between DBnary data and other lemon datasets. The prefix has been fixed in the extractor and I also fixed ALL PREVIOUS VERSIONS of the dataset. Meaning that from now on, even if you use an older dataset (provided that you re-download it), you’ll have the correct mapping.

Read More

Improvement in Russian extractor

Several bug fixes were made on the Russian extractor. Definitions should now be extracted in a more complete way and empty definitions should not be extracted anymore. Translation extraction has also been modified to avoid (rather unfrequent) special cases where links and usage notes where not extracted correctly.

Read More

French and German morphology is now extracted

Since December 2014, morphological data has been extracted from French and German language edition. This data is currently stored in exhaustive version, meaning that every inflected form may be found in an a lemon:otherForm property. German morphology extraction is still rather preliminar, but French version should already be in a form useful for DBnary users.

Read More

English extraction improved

English extraction has been improved slightly. More part of speeches are now extracted (mainly affixes, proverbs, …), Extracted data is now more precisely typed and described, using lexinfo vocabulary, Additionally, All lexical entries are now explicitly typed as LexicalEntry while they were previously typed as lemon:Word or lemon:Phrase or lemon:Adjective (all of which are sub classes of lemon:LexicalEntry).

Read More

Usage examples are now extracted and attached to word senses

During the French “TALN” workshop, several French researchers asked me if I could add usage examples in the extracted data. This has been done in French, with the addition of a new property (named dbnary:exampleSource) that gives the source of the example when available. The French extractor still shows some problems when examples are splitter on several lines using te HTML br tag, but this is a minor issue (it appears a than sands times on a set of more than 252 thousands usage examples).

Read More

Translations are now connected to word senses

A translation is supposed to connect a source word sense to a target word sense. However, current DBnary data connects a lexical entry to a target string. By using available glosses, we are able to connect translations to their source word sense. We evaluate the accuracy of these relations to over 80%.

Read More

Finnish translation extractor improved

The Finnish language edition recently used a new way to encode translations. Hence, the extractor’s performance was reduced (as may be seen in the Finnish Stats). The extrator has been adapted to deal with the new translation encoding and extraction rate is now back to normal.

Read More

Portuguese extractor enhanced

The Portuguese wiktionary extractor has been considerably enhanced. After adapted it to a new architecture using GWT-Wiki library, I fixed several bugs in the translation extraction section. Moreover, homonymous entries were not extracted correctly. Hence since July 9th dump, the number of extracted entries and translations has increased.

Read More