README
¶
go-conllu
CoNLL-U parser written in Go. Convert CoNLL-U files to in-memory structs
The Computational Natural Language Learning - U format (CoNLL-U) is used by the Universal Dependencies project to represent natural language annotations.
go-conllu parses CoNNL-U file formats and exposes the data via in-memory Go structs.
⚙️ Installation
go get github.com/nuvi/go-conllu
🚀 Quick Start
package main
import (
"fmt"
"log"
conllu "github.com/nuvi/go-conllu"
)
func main() {
sentences, err := conllu.ParseFile("../../test_data/en_ewt-ud-train.small.conllu")
if err != nil {
log.Fatal(err)
}
for _, sentence := range sentences {
for _, token := range sentence.Tokens {
fmt.Println(token)
}
fmt.Println()
}
}
Issues
All issues should be submitted via the issues tab on Github. Please provide the code and data used in order for us to reproduce the issue.
💬 Contact
Feel free to reach out with questions/comments to maintainers:
Transient Dependencies
None, and we plan to keep it that way.
👏 Contributing
We love help! Contribute by forking the repo and opening pull requests. Please ensure that your code passes the existing tests and linting processes, and write new tests to test your changes if applicable.
All pull requests should be submitted to the "master" branch.
go test
go fmt
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type MorphologicalFeature ¶
MorphologicalFeature from the universal feature inventory (https://universaldependencies.org/u/feat/index.html) or from a defined language-specific extension (https://universaldependencies.org/ext-feat-index.html)
type Sentence ¶
type Sentence struct {
// Tokens of the sentence
Tokens []Token
// The original sentence generated from the comment, if it exists
Text string
}
Sentence represents a single sentence of parsed CoNLL-U tokens
type Token ¶
type Token struct {
// Word index, float starting at 1 for each new sentence
// If a range was found on a single line in the file
// then nultiple tokens will be created
ID float64
// Word form or punctuation symbol
Form string
// Lemma or stem of word form
Lemma string
// Universal part-of-speech tag
UPOS string
// Language-specific part-of-speech tag; empty if not available
XPOS string
// List of morphological features, which are described on the type
// nil if not available
Feats []MorphologicalFeature
// Head of the current word
// Either the ID of the head token for this word, or 0 if root
// https://universaldependencies.org/format.html#syntactic-annotation
Head float64
// Universal dependency relation to the HEAD (root iff HEAD = 0)
// or a defined language-specific subtype of one
Deprel string
// Enhanced dependency graph in the form of a list of head-deprel pairs. See Dep type for more information; nil if none.
// Dependencies that are shared between the basic and the enhanced dependency representations must be repeated in the Deps field
Deps []Dep
// Any other annotation, nil if none.
// https://universaldependencies.org/format.html#miscellaneous
Misc []string
}
Token represents a single token, e.g. "hello", "bye" and holds all associated annotations https://universaldependencies.org/format.html#conll-u-format
Source Files
Directories