README
¶
cirrus
quick and dirty entity recognition
Cirrus attempts to serialize arbitrary natural text into a Go value using dictionary matching. At the moment it doesn't really do much, and shouldn't be used by anybody.
Cirrus tries to parse tokens into various values based on pretty simple heuristics like capitalization and the presence of a dollar sign. Some entities, like cardinality ("one", "many", "ten", etc) have to be matched against every token.
Roadmap
- grouping of sequential entities
- e.g. sequential cardinals can be grouped,
two dozenbecomesResult{Value: 24}
- e.g. sequential cardinals can be grouped,
- text classification
- use wordnet?
Examples
2021-10-22is adate$20is avalueinUSD20mphis avalueinmiles per hourHong Kongis acityCharles Dickensis apersonMicrosoftorMSFTis acompanyAustraliais acountry
Numbers
1.2
10e-3
1.00001
20E10
15
2/3
one
two thousand three hundred seventy five
Dates
Christmas eve
10/22/15
10.22.15
2005
december 5 2005 // variations thereof
Times
12:10
ten minutes and five seconds
Phone Numbers
Addresses
Email Addresses
- probably best to use an established regexp for this
Web Addresses
- best to use an established regexp or std's
url.Parse
Tables
| name | age |
| john | 2 |
| jane | 12 |
name,age
john,2
jane,12
name age
"john" 2
"jane" 12
name age
john 2
jane 12
Documentation
¶
Overview ¶
Classifier attempts to classify documents by topic.
Index ¶
Constants ¶
const ( NO_UNIT = iota // metric METERS INCHES FEET MILES TIME )
const ( NONE = iota LINK QUANTITY DATE ORG CARDINAL MONEY EVENT )
Variables ¶
var ( //go:embed data/cardinality.txt CARDINALITY_DICT_EMBED string CARDINALITY_DICT = LoadDictionary(CARDINALITY_DICT_EMBED) )
var (
ErrNoExtract = errors.New("couldn't determine meaning")
)
var NUM_REGEXP = regexp.MustCompile(`\d+`)
Matches any sequence of one or more numbers
var SINGLE_NUMBER_REGEXP = regexp.MustCompile(`\d`)
Matches a single number
var UNIT_EXTRACT_REGEXP = regexp.MustCompile(`\d+\s?[A-Z..a-z]+`)
Matches one or more numbers followed by optional whitespace and a sequence of one or more letters, used for determining whether a string may be a quantity or not.
var UNIT_TYPE_REGEXP = regexp.MustCompile(`[a-zA-Z]+`)
Matches one or more letters, used as a filter for determining the unit
var Units = [][]string{
NO_UNIT: {"none"},
METERS: {"meters", "m"},
INCHES: {"inches", "in"},
FEET: {"feet", "ft", "foot"},
MILES: {"mile", "mi"},
TIME: {"minute", "second", "hour", "day"},
}
Recognized units.
Functions ¶
func LoadDictionary ¶
Loads a dictionary from an embeded newline-delimited dictionary file. The dictionary words are placed in a trie.
Types ¶
type Result ¶
type Result struct {
// type of result
ResultType ResultType `json:"type"`
// title/label of the unit, as in with graphs
// e.g. "Number of Queries Per Second"
Label string `json:"label"`
// Custom data
Data dataType `json:"data"`
// value
Value string `json:"value"`
Start, End uint
}
type ResultType ¶
type ResultType int
func (ResultType) String ¶
func (r ResultType) String() string
Source Files