zannotate

package module
v1.0.0-rc1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 26, 2026 License: Apache-2.0 Imports: 31 Imported by: 0

README

ZAnnotate

ZAnnotate is a Go utility that facilitates annotating large IP datasets with network metadata. Right now this includes:

CLI Flag Description Needs API Key Needs Data Download
--censys Censys internet intelligence (live API) Yes
--cymru Cymru IP Origin/Peer ASN and ASN details
--geoasn MaxMind GeoIP ASN data Yes
--geoip2 MaxMind GeoIP2 city and geolocation data Yes
--greynoise GreyNoise Psychic threat intelligence and CVE data Yes (to download) Yes
--ipinfo IPInfo.io ASN and geolocation data Yes
--rdap RDAP (WHOIS successor) lookups (live)
--rdns Reverse DNS lookups (live)
--routing BGP/Routing data from an MRT routing table Yes
--spur Spur Intelligence (ASN, organization, infrastructure classification, geolocation) Yes

Jump to module setup: Censys · Cymru · GeoASN · GeoIP · GreyNoise · IPInfo.io · RDAP · RDNS/Reverse DNS · Routing/BGP · Spur

You can use any combination of the annotators, for example here is reverse DNS and IPInfo annotations together:

echo "1.1.1.1" | zannotate --rdns --ipinfo --ipinfo-database=./data-snapshots/ipinfo_lite.mmdb
{
   "ip":"1.1.1.1",
   "ipinfo":{"country":"Australia","country_code":"AU","continent":"Oceania","continent_code":"OC","asn":"AS13335","as_name":"Cloudflare, Inc.","as_domain":"cloudflare.com"},
   "rdns":{"domain_names":["one.one.one.one"]}
}

The --help has more details on each annotator and it's available flags

zannotate --help

Installation

ZAnnotate can be installed using make install

make install

or if you don't have make installed, you can use the following command:

cd cmd/zannotate && go install

Either way, this will install the zannotate binary in your $GOPATH/bin directory.

Check that it was installed correctly with:

zannotate --help

Input/Output

Input
New-line Separated IPs

By default, ZAnnotate expects new-line delimited IP addresses on standard input. For example:

printf "1.1.1.1\n8.8.8.8" | zannotate --rdns
{"ip":"1.1.1.1","rdns":{"domain_names":["one.one.one.one"]}}
{"ip":"8.8.8.8","rdns":{"domain_names":["dns.google"]}}
JSON

You may wish to annotate data that is already in JSON format. You'll then need to use the --input-file-type=json flag. This will insert a zannotate field into the existing JSON object. For example:

echo '{"ip": "1.1.1.1"}' | zannotate --rdns --geoasn --geoasn-database=/path-to-geo-asn.mmdb --input-file-type=json
{"ip":"1.1.1.1","zannotate":{"geoasn":{"asn":13335,"org":"CLOUDFLARENET"},"rdns":{"domain_names":["one.one.one.one"]}}}

If your JSON objects have a different field for the IP address than the default ip, you can specify that with the --input-ip-field flag. For example, if your JSON objects have an ip_address, you can use:

echo '{"ip_address": "1.1.1.1"}' | zannotate --rdns --input-file-type=json --input-ip-field=ip_address
{"ip_address":"1.1.1.1","zannotate":{"rdns":{"domain_names":["one.one.one.one"]}}}
CSV

If your input data is in CSV format, you can use the --input-file-type=csv flag.

printf "name,ip,date\n cloudflare,1.1.1.1,04-04-26\n google,8.8.8.8,04-04-26" | zannotate --rdns --input-file-type=csv
{"name":" cloudflare","ip":"1.1.1.1","date":"04-04-26","zannotate":{"rdns":{"domain_names":["one.one.one.one"]}}}
{"name":" google","ip":"8.8.8.8","date":"04-04-26","zannotate":{"rdns":{"domain_names":["dns.google"]}}}

Similar to JSON, you can use the --input-ip-field flag to specify a column other than ip that contains the IP address.

printf "name,ip_address,date\n cloudflare,1.1.1.1,04-04-26\n google,8.8.8.8,04-04-26" | zannotate --rdns --input-file-type=csv --input-ip-field=ip_address
{"date":"04-04-26","zannotate":{"rdns":{"domain_names":["dns.google"]}},"name":" google","ip_address":"8.8.8.8"}
{"date":"04-04-26","zannotate":{"rdns":{"domain_names":["one.one.one.one"]}},"name":" cloudflare","ip_address":"1.1.1.1"}
Output

By default, ZAnnotate reads new-line delimited IP addresses from standard input and outputs a JSON object per line to standard output like:

echo "1.1.1.1" | zannotate --rdns --geoasn --geoasn-database=/path-to-geo-asn.mmdb
{"ip":"1.1.1.1","geoasn":{"asn":13335,"org":"CLOUDFLARENET"},"rdns":{"domain_names":["one.one.one.one"]}}

If an IP address cannot be annotated, either because of an error or lack of data, there will be an empty field for that annotation. For example, if an IP address is private and therefore has no RDNS or ASN data, the output will look like:

echo "127.0.0.1" | zannotate --rdns --geoasn --geoasn-database=/path-to-geo-asn.mmdb
{"geoasn":{},"rdns":{},"ip":"127.0.0.1"}
JSON and CSV Output Flags

The --output-annotation-field flag can be used to specify a different field name for the annotations instead of zannotate for both CSV and JSON file inputs.

For example using the output tag --output-annotation-field="info" with JSON input:

printf "name,ip_address,date\n cloudflare,1.1.1.1,04-04-26\n google,8.8.8.8,04-04-26" | zannotate --rdns --input-file-type=csv --input-ip-field=ip_address --output-annotation-field="info"
{"name":" cloudflare","ip_address":"1.1.1.1","date":"04-04-26","info":{"rdns":{"domain_names":["one.one.one.one"]}}}
{"ip_address":"8.8.8.8","date":"04-04-26","info":{"rdns":{"domain_names":["dns.google"]}},"name":" google"}

Modules

[!NOTE] URLs and instructions may change over time. These are up-to-date as of May 2026.

Censys

Censys provides internet-wide host and network data, including information on what services are running on an IP, what TLS certificates it has, and more. They offer a free tier that allows for a limited number of queries per month, which can be used to enrich IP annotations with Censys data.

[!NOTE] The free account (as of April 2026) allows a single concurrent request and 100 requests per month. Once you've used all your credits, the Censys API will return an error upon further requests until your credits reset at the beginning of the next month or you purchase additional credits. ZAnnotate will quit with an error if this occurs to prevent silent errors, so if you have a large dataset to annotate, be mindful of your credit usage and prioritize only annotating the IPs you have credits for. With the free account only offering a single concurrent request, you'll want to leave --censys-threads=1 unless you pay for a higher tier.

  1. Create an account at Censys.io and get a Personal Access Token (PAT) from Personal Settings > Personal Access Tokens.
  2. Annotate with Censys data:
echo "8.8.8.8" | zannotate --censys --censys-pat="CENSYS_PAT_HERE"

Results truncated for brevity:

{
   "ip":"8.8.8.8",
   "censys": {
      "whois":{
         "network":{ "name":"Google LLC","cidrs":["8.8.8.0/24"],"created":"2023-12-28T00:00:00Z","updated":"2023-12-28T00:00:00Z","allocation_type":"ALLOCATION","handle":"GOGL"},
         "organization":{"state":"CA","postal_code":"94043","country":"US","tech_contacts":[{"handle":"ZG39-ARIN","name":"Google LLC","email":"arin-contact@google.com"}],
            "handle":"GOGL","street":"1600 Amphitheatre Parkway","abuse_contacts":[{"handle":"ABUSE5250-ARIN","name":"Abuse","email":"network-abuse@google.com"}],
            "admin_contacts":[{"handle":"ZG39-ARIN","name":"Google LLC","email":"arin-contact@google.com"}],
            "name":"Google LLC",
            "city":"Mountain View"}},
      "services":[{
         "port":53,"protocol":"DNS","transport_protocol":"udp","ip":"8.8.8.8","scan_time":"2026-04-10T02:55:53Z"}]}}
Cymru

Cymru provides ASN and BGP peering info for IP addresses. You can use the --cymru flag to query Cymru for this information. Cymru has setup a DNS service to provide this information and by default ZAnnotate will use your system default DNS resolvers to query. If you find you're seeing timeouts using your local DNS resolver, you can specify custom resolvers with the --cymru-dns-servers flag. --cymru-threads and --cymru-timeout are also available to configure performance of Cymru annotations, defaults are set to get good performance while avoiding timeouts with most DNS resolvers. By default, the Cymru annotator fetches the origin and peer ASs' as well as AS details on all ASNs. Should you desire a subset, the following flags are available: --cymru-annotate-origin-as, --cymru-annotate-peer-as, and --cymru-annotate-as-details

printf "1.1.1.1" | ./zannotate --cymru --cymru-dns-servers=1.1.1.1,1.0.0.1
{"ip":"207.243.195.103","cymru":{"origin_asns":[7018],"peer_asns":[6939,1299,2914,3257,6461,6762],"asn_details":{"7018":{"asn":7018,"country_code":"US","registry":"arin","asn_allocation_date":"1996-07-30","asn_description":"ATT-INTERNET4 - AT&T Enterprises, LLC, US"},"6939":{"asn":6939,"country_code":"US","registry":"arin","asn_allocation_date":"1996-06-28","asn_description":"HURRICANE - Hurricane Electric LLC, US"},"1299":{"asn":1299,"country_code":"SE","registry":"ripencc","asn_allocation_date":"1993-09-01","asn_description":"TWELVE99 Arelion, fka Telia Carrier, SE"},"2914":{"asn":2914,"country_code":"US","registry":"arin","asn_allocation_date":"1998-12-07","asn_description":"NTT-DATA-2914 - NTT America, Inc., US"},"3257":{"asn":3257,"country_code":"US","registry":"ripencc","asn_allocation_date":"1994-09-30","asn_description""GTT-BACKBONE GTT, US"},"6461":{"asn":6461,"country_code":"US","registry":"arin","asn_allocation_date":"1996-04-22","asn_description":"ZAYO-6461 - Zayo Bandwidth, US"},"6762":{"asn":6762,"country_code":"IT","registry":"ripencc","asn_allocation_date":"1996-09-12","asn_description":"SEABONE-NET TELECOM ITALIA SPARKLE S.p.A., IT"}},"prefix_details":{"207.242.0.0/15":{"prefix":"207.242.0.0/15","origin_asns":[7018],"peer_asns":[1299,2914,3257,6461,6762,6939],"country_code":"US","registry":"arin","allocation_date":"1996-11-01" }}}}
GreyNoise Psychic

GreyNoise is an IP intelligence feed that provides metadata like threat classification and associated CVEs. Their Psychic data downloads provide their data feed in a database suitable for offline data enrichment. To use their download with zannotate, you'll want to download an .mmdb formatted file using your GreyNoise API key. As of April 2026, signing up with a free account gives access to data downloads.

  1. Sign up for a free GreyNoise account here.
  2. Copy your API key from the appropriate section of your account.
  3. Download a mmdb file. The below command downloads data for a single date (April 7th, 2026). You can also download data for a range of days and for models of various levels of detail — see GreyNoise's Psychic documentation for more details.
curl -H "key: GREYNOISE_API_KEY_HERE" \
     https://psychic.labs.greynoise.io/v1/psychic/download/2026-04-07/3/mmdb \
     -o /tmp/m3.mmdb
  1. Test GreyNoise data enrichment:

[!NOTE] The below examples use the exact data download from the above curl command. What results you see will depend on the data downloaded.

echo "14.1.105.157" | zannotate --greynoise --greynoise-database=/tmp/m3.mmdb
{"greynoise":{"classification":"malicious","cves":["CVE-2015-2051","CVE-2016-20016","CVE-2018-10561","CVE-2018-10562","CVE-2016-6277","CVE-2024-12847"],"date":"2026-04-07","handshake_complete":true,"last_seen":"2026-04-07T00:00:00Z","seen":true,"tags":["Mirai TCP Scanner","Mirai","Telnet Protocol","Generic IoT Default Password Attempt","Web Crawler","Generic Suspicious Linux Command in Request","HNAP Crawler","Telnet Login Attempt","D-Link Devices HNAP SOAPAction Header RCE Attempt","MVPower CCTV DVR RCE CVE-2016-20016 Attempt","JAWS Webserver RCE","GPON CVE-2018-10561 Router Worm","Generic ${IFS} Use in RCE Attempt","CCTV-DVR RCE","NETGEAR Command Injection CVE-2016-6277","NETGEAR DGN setup.cgi CVE-2024-12847 Command Execution Attempt","CGI Script Scanner"],"actor":"unknown"},"ip":"14.1.105.157"}

Note that many IPs will not be in the GreyNoise dataset, so you may see output like the following:

echo "1.1.1.1" | zannotate --greynoise --greynoise-database=/tmp/m3.mmdb
{"ip":"1.1.1.1","greynoise":{}}
IPInfo.io

IPInfo.io provides a free dataset that includes ASN and geolocation data, scoped to the country level. Paid tiers provide more granular geolocation data.

  1. Sign up for a free account at IPInfo.io.
  2. Navigate to the Data Download page
  3. Download the mmdb file IPInfo Download Page
  4. Example CLI usage
echo "1.1.1.1" | zannotate --ipinfo --ipinfo-database=./path-to-ipinfo-db.mmdb
{"ip":"1.1.1.1","ipinfo":{"country":"Australia","country_code":"AU","continent":"Oceania","continent_code":"OC","asn":"AS13335","as_name":"Cloudflare, Inc.","as_domain":"cloudflare.com"}}
MaxMind GeoIP ASN and City (Formerly GeoIP2)

MaxMind provides datasets for IP geolocation and ASN data in both a free (GeoLite) and paid (GeoIP) version. Additionally, both the GeoLite and GeoIP datasets come in two access patterns - a downloadable database file that can be queried locally and a web API. The GeoIP module in ZAnnotate supports the local database in both GeoLite and GeoIP versions.

The following assumes you want to use the free GeoLite datasets, but the process is similar for the paid GeoIP data.

  1. Sign-up form for MaxMind GeoLite Access
  2. Login to your account
  3. Go to the "GeoIP / GeoLite" > "Download files" section and download the zip files for either GeoLite ASN or GeoLite City datasets.

GeoLite Download Page

  1. Unzip, place the .mmdb files somewhere and test with the below.
MaxMind GeoIP City
echo "171.67.71.209" | zannotate --geoip2 --geoip2-database=./path-to-geolite2.mmdb
{
   "ip":"171.67.71.209",
   "geoip2":{
      "city":{"name":"Vallejo","id":5405380},
      "country":{"name":"United States","code":"US","id":6252001},
      "continent":{"name":"North America","code":"NA","id":6255149},
      "postal":{"code":"94590"},
      "latlong":{"accuracy_radius":50,"latitude":38.1043,"longitude":-122.2442,"metro_code":807,"time_zone":"America/Los_Angeles"},
      "represented_country":{},
      "registered_country":{"name":"United States","code":"US","id":6252001},
      "metadata":{}}
}
MaxMind GeoLite ASN
echo "171.67.71.209" | zannotate --geoasn --geoasn-database=/path-to-asn-file.mmdb
{"ip":"171.67.71.209","geoasn":{"asn":32,"org":"STANFORD"}}
RDAP (WHOIS successor)

RDAP (Registration Data Access Protocol) is a protocol used to access registration data for internet resources such as IP addresses and domain names and is the successor to WHOIS. ZAnnotate can query RDAP servers to pull registration data for IPs.

echo "1.1.1.1" | zannotate --rdap

Results truncated for brevity:

{
   "ip":"1.1.1.1",
   "whois": {
     "DecodeData": {},
     "Lang": "",
     "Conformance": [
       "history_version_0",
       "nro_rdap_profile_0",
       "cidr0",
       "rdap_level_0"
     ],
     "ObjectClassName": "ip network"
   }
}

This should give you the same output as a direct query to an RDAP server, for example:

rdap 1.1.1.1
IP Network:
  Handle: 1.1.1.0 - 1.1.1.255
  Start Address: 1.1.1.0
  End Address: 1.1.1.255
  IP Version: v4
  Name: APNIC-LABS
  Type: ASSIGNED PORTABLE
...<further output truncated for brevity>
Reverse DNS (RDNS)

ZAnnotate can perform reverse DNS lookups for each IP address. No data download is required, --rdns queries live DNS servers directly.

printf "1.1.1.1\n8.8.8.8" | zannotate --rdns
{"ip":"1.1.1.1","rdns":{"domain_names":["one.one.one.one"]}}
{"ip":"8.8.8.8","rdns":{"domain_names":["dns.google"]}}

If an IP doesn't have a PTR record, the rdns field will be empty:

echo "127.0.0.1" | zannotate --rdns
{"ip":"127.0.0.1","rdns":{}}
BGP Routing Tables
  1. Go to https://archive.routeviews.org/route-views2/bgpdata/
  2. Select a month directory (e.g. 2025.09)
  3. Select the RIBS/ directory
  4. Download a zipped MRT file (e.g. rib.20250923.1200.bz2)
  5. Unzip the file with:
bzip2 -d ./path-to-downloaded-file/rib.20250923.1200.bz2
  1. Test with:
echo "1.1.1.1" | zannotate --routing --routing-mrt-file=/tmp/rib.20250923.1200
{"ip":"1.1.1.1","routing":{"prefix":"1.1.1.0/24","asn":13335,"path":[3561,209,3356,13335]}}
Spur IP Enrichment + Intelligence

spur.us provides per-IP intelligence such as ASN and organization, infrastructure classification (e.g., datacenter, CDN, mobile), and geolocation metadata. We can query spur.us alongside other sources to enrich annotations and help identify datacenter/Anycast deployments, CDNs, and ISP ownership.

  1. Get an API key from Spur. Depending on current pricing, you may need to sign up for a paid account — check spur.us/pricing for details.
  2. Set your API key as an environment variable:
export SPUR_API_KEY=your_api_key_here

(If you want to make this permanent, add the above line to your shell profile, e.g. ~/.bashrc or ~/.zshrc)

  1. Test with:
echo "1.1.1.1" | zannotate --spur
{"ip":"1.1.1.1","spur":{"as":{"number":13335,"organization":"Cloudflare, Inc."},"infrastructure":"DATACENTER","ip":"1.1.1.1","location":{"city":"Anycast","country":"ZZ","state":"Anycast"},"organization":"Taguchi Digital Marketing System"}}

Documentation

Index

Constants

View Source
const SpurApiUrl = "https://api.spur.us/v2/context/"

Variables

View Source
var Annotators []AnnotatorFactory

Functions

func AnnotateInputDecode

func AnnotateInputDecode(conf *GlobalConf, inChan <-chan string,
	outChan chan<- inProcessIP, wg *sync.WaitGroup, i int)

multiple workers that decode raw lines from AnnotateRead from JSON/CSV into native golang objects

func AnnotateOutputEncode

func AnnotateOutputEncode(conf *GlobalConf, inChan <-chan inProcessIP,
	outChan chan<- string, wg *sync.WaitGroup, i int)

func AnnotateRead

func AnnotateRead(conf *GlobalConf, path string, in chan<- string)

single worker that reads from file and queues raw lines

func AnnotateWorker

func AnnotateWorker(conf *GlobalConf, a Annotator, inChan <-chan inProcessIP,
	outChan chan<- inProcessIP, fieldName string, wg *sync.WaitGroup, i int)

func AnnotateWrite

func AnnotateWrite(path string, out <-chan string, wg *sync.WaitGroup)

func DoAnnotation

func DoAnnotation(conf *GlobalConf)

func PerSecondUpdateWorker

func PerSecondUpdateWorker(filePath string, outChan <-chan string, wg *sync.WaitGroup)

PerSecondUpdateWorker prints a per-second scan summary as well as a Scan Completed/Aborted msg at the end It writes the updates to the file path provided, or stderr if the file path is empty or "-". For every line of output received on outChan, it counts one IP annotated

func RegisterAnnotator

func RegisterAnnotator(af AnnotatorFactory)

Types

type ASNLookup

type ASNLookup struct {
	ASN               uint32 `json:"asn,omitempty"`
	CountryCode       string `json:"country_code,omitempty"`
	Registry          string `json:"registry,omitempty"`
	ASNAllocationDate string `json:"asn_allocation_date,omitempty"`
	ASNDescription    string `json:"asn_description,omitempty"`
}

ASNLookup contains the result of a query to ASX.asn.cymru.com

type Annotator

type Annotator interface {
	Initialize() error
	Annotate(ip net.IP) interface{}
	GetFieldName() string
	Close() error
}

type AnnotatorFactory

type AnnotatorFactory interface {
	Initialize(c *GlobalConf) error
	AddFlags(flags *flag.FlagSet)
	GroupName() string
	GetWorkers() int
	IsEnabled() bool
	MakeAnnotator(i int) Annotator
	Close() error
}

type BasePluginConf

type BasePluginConf struct {
	Threads int
	Enabled bool
}

type CensysAnnotator

type CensysAnnotator struct {
	Factory *CensysAnnotatorFactory
	Id      int
}

CensysAnnotator (Per-Worker)

func (*CensysAnnotator) Annotate

func (a *CensysAnnotator) Annotate(ip net.IP) interface{}

Annotate performs a Censys host lookup for the given IP address and returns the results. If an error occurs or a lookup fails, it returns nil

func (*CensysAnnotator) Close

func (a *CensysAnnotator) Close() error

func (*CensysAnnotator) GetFieldName

func (a *CensysAnnotator) GetFieldName() string

func (*CensysAnnotator) Initialize

func (a *CensysAnnotator) Initialize() (err error)

type CensysAnnotatorFactory

type CensysAnnotatorFactory struct {
	BasePluginConf
	// contains filtered or unexported fields
}

func (*CensysAnnotatorFactory) AddFlags

func (a *CensysAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*CensysAnnotatorFactory) Close

func (a *CensysAnnotatorFactory) Close() error

func (*CensysAnnotatorFactory) GetWorkers

func (a *CensysAnnotatorFactory) GetWorkers() int

func (*CensysAnnotatorFactory) GroupName

func (a *CensysAnnotatorFactory) GroupName() string

func (*CensysAnnotatorFactory) Initialize

func (a *CensysAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*CensysAnnotatorFactory) IsEnabled

func (a *CensysAnnotatorFactory) IsEnabled() bool

func (*CensysAnnotatorFactory) MakeAnnotator

func (a *CensysAnnotatorFactory) MakeAnnotator(i int) Annotator

type CymruAnnotator

type CymruAnnotator struct {
	Factory *CymruAnnotatorFactory
	Id      int
	// contains filtered or unexported fields
}

func (*CymruAnnotator) Annotate

func (a *CymruAnnotator) Annotate(ip net.IP) interface{}

Annotate performs a Cymru data lookup for the given IP address and returns the results. If an error occurs or a lookup fails, it returns nil

func (*CymruAnnotator) Close

func (a *CymruAnnotator) Close() error

func (*CymruAnnotator) GetFieldName

func (a *CymruAnnotator) GetFieldName() string

func (*CymruAnnotator) Initialize

func (a *CymruAnnotator) Initialize() (err error)

Cymru Annotator (Per-Worker)

type CymruAnnotatorFactory

type CymruAnnotatorFactory struct {
	BasePluginConf
	RawResolvers string
	// contains filtered or unexported fields
}

func (*CymruAnnotatorFactory) AddFlags

func (a *CymruAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*CymruAnnotatorFactory) Close

func (a *CymruAnnotatorFactory) Close() error

func (*CymruAnnotatorFactory) GetWorkers

func (a *CymruAnnotatorFactory) GetWorkers() int

func (*CymruAnnotatorFactory) GroupName

func (a *CymruAnnotatorFactory) GroupName() string

func (*CymruAnnotatorFactory) Initialize

func (a *CymruAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*CymruAnnotatorFactory) IsEnabled

func (a *CymruAnnotatorFactory) IsEnabled() bool

func (*CymruAnnotatorFactory) MakeAnnotator

func (a *CymruAnnotatorFactory) MakeAnnotator(i int) Annotator

type CymruResult

type CymruResult struct {
	OriginASNs []uint32     `json:"origin_asns,omitempty"`
	PeerASNs   []uint32     `json:"peer_asns,omitempty"`
	ASNLookup  []*ASNLookup `json:"asn_details,omitempty"` // both Peer and Origin ASN Details

	PrefixDetails []*PrefixResult `json:"prefix_details,omitempty"` // Prefix to details
	// contains filtered or unexported fields
}

CymruResult stores the format for the result from the Cymru annotator

type GeoIP2Annotator

type GeoIP2Annotator struct {
	Factory *GeoIP2AnnotatorFactory
	Reader  *geoip2.Reader
	Id      int
}

func (*GeoIP2Annotator) Annotate

func (a *GeoIP2Annotator) Annotate(ip net.IP) interface{}

func (*GeoIP2Annotator) Close

func (a *GeoIP2Annotator) Close() error

func (*GeoIP2Annotator) GeoIP2FillStruct

func (a *GeoIP2Annotator) GeoIP2FillStruct(in *geoip2.City) *GeoIP2Output

func (*GeoIP2Annotator) GetFieldName

func (a *GeoIP2Annotator) GetFieldName() string

func (*GeoIP2Annotator) Initialize

func (a *GeoIP2Annotator) Initialize() error

type GeoIP2AnnotatorFactory

type GeoIP2AnnotatorFactory struct {
	BasePluginConf
	Path       string
	Mode       string
	Language   string
	RawInclude string

	Conf *GlobalConf
	// what data to include
	IncludeCity               bool
	IncludeCountry            bool
	IncludeContinent          bool
	IncludePostal             bool
	IncludeLatLong            bool
	IncludeTraits             bool
	IncludeSubdivisions       bool
	IncludeRepresentedCountry bool
	IncludeRegisteredCountry  bool
}

func (*GeoIP2AnnotatorFactory) AddFlags

func (a *GeoIP2AnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*GeoIP2AnnotatorFactory) Close

func (a *GeoIP2AnnotatorFactory) Close() error

func (*GeoIP2AnnotatorFactory) GetWorkers

func (a *GeoIP2AnnotatorFactory) GetWorkers() int

func (*GeoIP2AnnotatorFactory) GroupName

func (a *GeoIP2AnnotatorFactory) GroupName() string

GeoIP2 Annotator Factory (Global)

func (*GeoIP2AnnotatorFactory) Initialize

func (a *GeoIP2AnnotatorFactory) Initialize(conf *GlobalConf) error

func (*GeoIP2AnnotatorFactory) IsEnabled

func (a *GeoIP2AnnotatorFactory) IsEnabled() bool

func (*GeoIP2AnnotatorFactory) MakeAnnotator

func (a *GeoIP2AnnotatorFactory) MakeAnnotator(i int) Annotator

type GeoIP2City

type GeoIP2City struct {
	Name      string `json:"name,omitempty"`
	GeoNameId uint   `json:"id,omitempty"`
}

type GeoIP2Country

type GeoIP2Country struct {
	Name      string `json:"name,omitempty"`
	Code      string `json:"code,omitempty"`
	GeoNameId uint   `json:"id,omitempty"`
}

type GeoIP2LatLong

type GeoIP2LatLong struct {
	AccuracyRadius uint16  `json:"accuracy_radius,omitempty"`
	Latitude       float64 `json:"latitude,omitempty"`
	Longitude      float64 `json:"longitude,omitempty"`
	MetroCode      uint    `json:"metro_code,omitempty"`
	TimeZone       string  `json:"time_zone,omitempty"`
}

type GeoIP2Output

type GeoIP2Output struct {
	City               *GeoIP2City    `json:"city,omitempty"`
	Country            *GeoIP2Country `json:"country,omitempty"`
	Continent          *GeoIP2Country `json:"continent,omitempty"`
	Postal             *GeoIP2Postal  `json:"postal,omitempty"`
	LatLong            *GeoIP2LatLong `json:"latlong,omitempty"`
	RepresentedCountry *GeoIP2Country `json:"represented_country,omitempty"`
	RegisteredCountry  *GeoIP2Country `json:"registered_country,omitempty"`
	Traits             *GeoIP2Traits  `json:"metadata,omitempty"`
}

type GeoIP2Postal

type GeoIP2Postal struct {
	Code string `json:"code,omitempty"`
}

type GeoIP2Traits

type GeoIP2Traits struct {
	IsAnonymousProxy    bool `json:"is_anonymous_proxy,omitempty"`
	IsSatelliteProvider bool `json:"is_satellite_provider,omitempty"`
}

type GeoIPASNAnnotator

type GeoIPASNAnnotator struct {
	Factory *GeoIPASNAnnotatorFactory
	Reader  *geoip2.Reader
	Id      int
}

func (*GeoIPASNAnnotator) Annotate

func (anno *GeoIPASNAnnotator) Annotate(ip net.IP) interface{}

func (*GeoIPASNAnnotator) Close

func (anno *GeoIPASNAnnotator) Close() error

func (*GeoIPASNAnnotator) GetFieldName

func (anno *GeoIPASNAnnotator) GetFieldName() string

func (*GeoIPASNAnnotator) Initialize

func (anno *GeoIPASNAnnotator) Initialize() error

type GeoIPASNAnnotatorFactory

type GeoIPASNAnnotatorFactory struct {
	BasePluginConf
	Path string
	Mode string
}

func (*GeoIPASNAnnotatorFactory) AddFlags

func (fact *GeoIPASNAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*GeoIPASNAnnotatorFactory) Close

func (fact *GeoIPASNAnnotatorFactory) Close() error

func (*GeoIPASNAnnotatorFactory) GetWorkers

func (fact *GeoIPASNAnnotatorFactory) GetWorkers() int

func (*GeoIPASNAnnotatorFactory) GroupName

func (fact *GeoIPASNAnnotatorFactory) GroupName() string

func (*GeoIPASNAnnotatorFactory) Initialize

func (fact *GeoIPASNAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*GeoIPASNAnnotatorFactory) IsEnabled

func (fact *GeoIPASNAnnotatorFactory) IsEnabled() bool

func (*GeoIPASNAnnotatorFactory) MakeAnnotator

func (fact *GeoIPASNAnnotatorFactory) MakeAnnotator(i int) Annotator

type GeoIPASNOutput

type GeoIPASNOutput struct {
	ASN    uint   `json:"asn,omitempty"`
	ASNOrg string `json:"org,omitempty"`
}

type GlobalConf

type GlobalConf struct {
	InputFilePath             string
	InputFileType             string
	OutputFilePath            string
	LogFilePath               string
	StatusUpdatesFilePath     string
	Verbosity                 int
	Threads                   int
	InputIPFieldName          string
	OutputAnnotationFieldName string
	InputDecodeThreads        int
	OutputEncodeThreads       int
	// contains filtered or unexported fields
}

type GreyNoiseAnnotator

type GreyNoiseAnnotator struct {
	Factory *GreyNoiseAnnotatorFactory
	Id      int
}

GreyNoiseAnnotator (Per-Worker)

func (*GreyNoiseAnnotator) Annotate

func (a *GreyNoiseAnnotator) Annotate(ip net.IP) interface{}

Annotate performs a GreyNoise local lookup for the given IP address and returns the results. If an error occurs or a lookup fails, it returns nil

func (*GreyNoiseAnnotator) Close

func (a *GreyNoiseAnnotator) Close() error

func (*GreyNoiseAnnotator) GetFieldName

func (a *GreyNoiseAnnotator) GetFieldName() string

func (*GreyNoiseAnnotator) Initialize

func (a *GreyNoiseAnnotator) Initialize() (err error)

type GreyNoiseAnnotatorFactory

type GreyNoiseAnnotatorFactory struct {
	BasePluginConf
	DBPath string // path to the .mmdb path
	// contains filtered or unexported fields
}

func (*GreyNoiseAnnotatorFactory) AddFlags

func (a *GreyNoiseAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*GreyNoiseAnnotatorFactory) Close

func (a *GreyNoiseAnnotatorFactory) Close() error

func (*GreyNoiseAnnotatorFactory) GetWorkers

func (a *GreyNoiseAnnotatorFactory) GetWorkers() int

func (*GreyNoiseAnnotatorFactory) GroupName

func (a *GreyNoiseAnnotatorFactory) GroupName() string

func (*GreyNoiseAnnotatorFactory) Initialize

func (a *GreyNoiseAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*GreyNoiseAnnotatorFactory) IsEnabled

func (a *GreyNoiseAnnotatorFactory) IsEnabled() bool

func (*GreyNoiseAnnotatorFactory) MakeAnnotator

func (a *GreyNoiseAnnotatorFactory) MakeAnnotator(i int) Annotator

type IPInfoAnnotator

type IPInfoAnnotator struct {
	Factory *IPInfoAnnotatorFactory
	Id      int
}

func (*IPInfoAnnotator) Annotate

func (a *IPInfoAnnotator) Annotate(inputIP net.IP) interface{}

func (*IPInfoAnnotator) Close

func (a *IPInfoAnnotator) Close() error

func (*IPInfoAnnotator) GetFieldName

func (a *IPInfoAnnotator) GetFieldName() string

func (*IPInfoAnnotator) Initialize

func (a *IPInfoAnnotator) Initialize() error

type IPInfoAnnotatorFactory

type IPInfoAnnotatorFactory struct {
	BasePluginConf
	DatabaseFilePath string
	// contains filtered or unexported fields
}

func (*IPInfoAnnotatorFactory) AddFlags

func (a *IPInfoAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*IPInfoAnnotatorFactory) Close

func (a *IPInfoAnnotatorFactory) Close() error

func (*IPInfoAnnotatorFactory) GetWorkers

func (a *IPInfoAnnotatorFactory) GetWorkers() int

func (*IPInfoAnnotatorFactory) GroupName

func (a *IPInfoAnnotatorFactory) GroupName() string

func (*IPInfoAnnotatorFactory) Initialize

func (a *IPInfoAnnotatorFactory) Initialize(conf *GlobalConf) (err error)

func (*IPInfoAnnotatorFactory) IsEnabled

func (a *IPInfoAnnotatorFactory) IsEnabled() bool

func (*IPInfoAnnotatorFactory) MakeAnnotator

func (a *IPInfoAnnotatorFactory) MakeAnnotator(i int) Annotator

type IPInfoMMDBOutput

type IPInfoMMDBOutput struct {
	City              string `maxminddb:"city"`
	Region            string `maxminddb:"region"`
	RegionCode        string `maxminddb:"region_code"`
	Country           string `maxminddb:"country"`
	CountryCode       string `maxminddb:"country_code"`
	Continent         string `maxminddb:"continent"`
	ContinentCode     string `maxminddb:"continent_code"`
	Latitude          string `maxminddb:"latitude"`
	Longitude         string `maxminddb:"longitude"`
	Timezone          string `maxminddb:"timezone"`
	PostalCode        string `maxminddb:"postal_code"`
	DMACode           string `maxminddb:"dma_code"`
	GeonameID         string `maxminddb:"geoname_id"`  // GeoNames database identifier (if available).
	Radius            string `maxminddb:"radius"`      // Accuracy radius in kilometers (if available).
	GeoChanged        string `maxminddb:"geo_changed"` // Timestamp or flag indicating when the geolocation last changed (if available).
	ASN               string `maxminddb:"asn"`
	ASName            string `maxminddb:"as_name"`
	ASDomain          string `maxminddb:"as_domain"`
	ASType            string `maxminddb:"as_type"`
	ASChanged         string `maxminddb:"as_changed"`
	CarrierName       string `maxminddb:"carrier_name"` // Name of the mobile carrier (if available).
	MobileCountryCode string `maxminddb:"mcc"`
	MobileNetworkCode string `maxminddb:"mnc"`
	PrivacyName       string `maxminddb:"privacy_name"` // Specific name of the privacy or anonymization service detected (e.g., “NordVPN”).
	IsProxy           string `maxminddb:"is_proxy"`
	IsRelay           string `maxminddb:"is_relay"`     // Boolean flag indicating use of a general relay service
	IsTOR             string `maxminddb:"is_tor"`       // Whether the IP is a known TOR exit node.
	IsVPN             string `maxminddb:"is_vpn"`       // Flag indicating use of a VPN Service
	IsAnonymous       string `maxminddb:"is_anonymous"` // True if the IP is associated with VPN, proxy, Tor, or a relay service.
	IsAnycast         string `maxminddb:"is_anycast"`   // Whether the IP is using anycast routing.
	IsHosting         string `maxminddb:"is_hosting"`   // True if the IP address is an internet service hosting IP address
	IsMobile          string `maxminddb:"is_mobile"`    // True if the IP address is associated with a mobile network or carrier.
	IsSatellite       string `maxminddb:"is_satellite"` // True if the IP address is associated with a satellite connection
}

IPInfoMMDBOutput includes both the Plus/Core/Lite IPInfo fields and their maxminddb tags. We'll convert this into a IPInfoModuleOutput for JSON output (converting string fields to appropriate types).

func (*IPInfoMMDBOutput) ToModuleOutput

func (in *IPInfoMMDBOutput) ToModuleOutput() *IPInfoModuleOutput

type IPInfoModuleOutput

type IPInfoModuleOutput struct {
	City              string  `json:"city,omitempty"`
	Region            string  `json:"region,omitempty"`
	RegionCode        string  `json:"region_code,omitempty"`
	Country           string  `json:"country,omitempty"`
	CountryCode       string  `json:"country_code,omitempty"`
	Continent         string  `json:"continent,omitempty"`
	ContinentCode     string  `json:"continent_code,omitempty"`
	Latitude          float64 `json:"latitude,omitempty"`
	Longitude         float64 `json:"longitude,omitempty"`
	Timezone          string  `json:"timezone,omitempty"`
	PostalCode        string  `json:"postal_code,omitempty"`
	DMACode           string  `json:"dma_code,omitempty"`    // Nielsen Designated Market Area code (if available).
	GeonameID         uint64  `json:"geoname_id,omitempty"`  // GeoNames database identifier (if available).
	Radius            uint64  `json:"radius,omitempty"`      // Accuracy radius in kilometers (if available).
	GeoChanged        string  `json:"geo_changed,omitempty"` // Timestamp or flag indicating when the geolocation last changed (if available).
	ASN               string  `json:"asn,omitempty"`
	ASName            string  `json:"as_name,omitempty"`
	ASDomain          string  `json:"as_domain,omitempty"`
	ASType            string  `json:"as_type,omitempty"`
	ASChanged         string  `json:"as_changed,omitempty"`
	CarrierName       string  `json:"carrier_name,omitempty"` // Name of the mobile carrier (if available).
	MobileCountryCode string  `json:"mobile_country_code,omitempty"`
	MobileNetworkCode string  `json:"mobile_network_code,omitempty"`
	PrivacyName       string  `json:"privacy_name,omitempty"` // Specific name of the privacy or anonymization service detected (e.g., “NordVPN”).
	IsProxy           *bool   `json:"is_proxy,omitempty"`
	IsRelay           *bool   `json:"is_relay,omitempty"`     // Boolean flag indicating use of a general relay service
	IsTOR             *bool   `json:"is_tor,omitempty"`       // Whether the IP is a known TOR exit node.
	IsVPN             *bool   `json:"is_vpn,omitempty"`       // Flag indicating use of a VPN Service
	IsAnonymous       *bool   `json:"is_anonymous,omitempty"` // True if the IP is associated with VPN, proxy, Tor, or a relay service.
	IsAnycast         *bool   `json:"is_anycast,omitempty"`   // Whether the IP is using anycast routing.
	IsHosting         *bool   `json:"is_hosting,omitempty"`   // True if the IP address is an internet service hosting IP address
	IsMobile          *bool   `json:"is_mobile,omitempty"`    // True if the IP address is associated with a mobile network or carrier.
	IsSatellite       *bool   `json:"is_satellite,omitempty"` // True if the IP address is associated with a satellite connection
}

IPInfoModuleOutput is the final output struct with appropriate types for JSON output

type PrefixResult

type PrefixResult struct {
	Prefix         string   `json:"prefix,omitempty"`
	OriginASNs     []uint32 `json:"origin_asns,omitempty"`
	PeerASNs       []uint32 `json:"peer_asns,omitempty"`
	CountryCode    string   `json:"country_code,omitempty"`
	Registry       string   `json:"registry,omitempty"`
	AllocationDate string   `json:"allocation_date,omitempty"`
}

type RDAPAnnotator

type RDAPAnnotator struct {
	Factory *RDAPAnnotatorFactory
	Id      int
	// contains filtered or unexported fields
}

func (*RDAPAnnotator) Annotate

func (a *RDAPAnnotator) Annotate(ip net.IP) interface{}

func (*RDAPAnnotator) Close

func (a *RDAPAnnotator) Close() error

func (*RDAPAnnotator) GetFieldName

func (a *RDAPAnnotator) GetFieldName() string

func (*RDAPAnnotator) Initialize

func (a *RDAPAnnotator) Initialize() error

type RDAPAnnotatorFactory

type RDAPAnnotatorFactory struct {
	BasePluginConf
	Timeout int // Timeout for each RDAP query, in seconds
}

func (*RDAPAnnotatorFactory) AddFlags

func (a *RDAPAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*RDAPAnnotatorFactory) Close

func (a *RDAPAnnotatorFactory) Close() error

func (*RDAPAnnotatorFactory) GetWorkers

func (a *RDAPAnnotatorFactory) GetWorkers() int

func (*RDAPAnnotatorFactory) GroupName

func (a *RDAPAnnotatorFactory) GroupName() string

RDAP Annotator Factory (Global)

func (*RDAPAnnotatorFactory) Initialize

func (a *RDAPAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*RDAPAnnotatorFactory) IsEnabled

func (a *RDAPAnnotatorFactory) IsEnabled() bool

func (*RDAPAnnotatorFactory) MakeAnnotator

func (a *RDAPAnnotatorFactory) MakeAnnotator(i int) Annotator

type RDNSAnnotator

type RDNSAnnotator struct {
	Factory *RDNSAnnotatorFactory
	Id      int
	// contains filtered or unexported fields
}

func (*RDNSAnnotator) Annotate

func (a *RDNSAnnotator) Annotate(ip net.IP) interface{}

Annotate performs a reverse DNS lookup for the given IP address and returns the results. If an error occurs or a lookup fails, it returns nil

func (*RDNSAnnotator) Close

func (a *RDNSAnnotator) Close() error

func (*RDNSAnnotator) GetFieldName

func (a *RDNSAnnotator) GetFieldName() string

func (*RDNSAnnotator) Initialize

func (a *RDNSAnnotator) Initialize() (err error)

type RDNSAnnotatorFactory

type RDNSAnnotatorFactory struct {
	BasePluginConf
	RawResolvers string
	// contains filtered or unexported fields
}

func (*RDNSAnnotatorFactory) AddFlags

func (a *RDNSAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*RDNSAnnotatorFactory) Close

func (a *RDNSAnnotatorFactory) Close() error

func (*RDNSAnnotatorFactory) GetWorkers

func (a *RDNSAnnotatorFactory) GetWorkers() int

func (*RDNSAnnotatorFactory) GroupName

func (a *RDNSAnnotatorFactory) GroupName() string

func (*RDNSAnnotatorFactory) Initialize

func (a *RDNSAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*RDNSAnnotatorFactory) IsEnabled

func (a *RDNSAnnotatorFactory) IsEnabled() bool

func (*RDNSAnnotatorFactory) MakeAnnotator

func (a *RDNSAnnotatorFactory) MakeAnnotator(i int) Annotator

type RDNSOutput

type RDNSOutput struct {
	DomainNames []string `json:"domain_names,omitempty"`
}

type RoutingAnnotator

type RoutingAnnotator struct {
	Factory *RoutingAnnotatorFactory

	Id int
	// contains filtered or unexported fields
}

func (*RoutingAnnotator) Annotate

func (a *RoutingAnnotator) Annotate(ip net.IP) interface{}

func (*RoutingAnnotator) Close

func (a *RoutingAnnotator) Close() error

func (*RoutingAnnotator) GetFieldName

func (a *RoutingAnnotator) GetFieldName() string

func (*RoutingAnnotator) Initialize

func (a *RoutingAnnotator) Initialize() error

type RoutingAnnotatorFactory

type RoutingAnnotatorFactory struct {
	BasePluginConf
	RoutingTablePath string
	ASNamesPath      string
	ASDataPath       string
	// contains filtered or unexported fields
}

func (*RoutingAnnotatorFactory) AddFlags

func (a *RoutingAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*RoutingAnnotatorFactory) Close

func (a *RoutingAnnotatorFactory) Close() error

func (*RoutingAnnotatorFactory) GetWorkers

func (a *RoutingAnnotatorFactory) GetWorkers() int

func (*RoutingAnnotatorFactory) GroupName

func (a *RoutingAnnotatorFactory) GroupName() string

Routing Annotator Factory (Global)

func (*RoutingAnnotatorFactory) Initialize

func (a *RoutingAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*RoutingAnnotatorFactory) IsEnabled

func (a *RoutingAnnotatorFactory) IsEnabled() bool

func (*RoutingAnnotatorFactory) MakeAnnotator

func (a *RoutingAnnotatorFactory) MakeAnnotator(i int) Annotator

type SpurAnnotator

type SpurAnnotator struct {
	Factory *SpurAnnotatorFactory
	Id      int
	// contains filtered or unexported fields
}

func (*SpurAnnotator) Annotate

func (a *SpurAnnotator) Annotate(ip net.IP) interface{}

Annotate performs a Spur data lookup for the given IP address and returns the results. If an error occurs or a lookup fails, it returns nil

func (*SpurAnnotator) Close

func (a *SpurAnnotator) Close() error

func (*SpurAnnotator) GetFieldName

func (a *SpurAnnotator) GetFieldName() string

func (*SpurAnnotator) Initialize

func (a *SpurAnnotator) Initialize() error

Spur Annotator (Per-Worker)

type SpurAnnotatorFactory

type SpurAnnotatorFactory struct {
	BasePluginConf
	// contains filtered or unexported fields
}

func (*SpurAnnotatorFactory) AddFlags

func (a *SpurAnnotatorFactory) AddFlags(flags *flag.FlagSet)

func (*SpurAnnotatorFactory) Close

func (a *SpurAnnotatorFactory) Close() error

func (*SpurAnnotatorFactory) GetWorkers

func (a *SpurAnnotatorFactory) GetWorkers() int

func (*SpurAnnotatorFactory) GroupName

func (a *SpurAnnotatorFactory) GroupName() string

func (*SpurAnnotatorFactory) Initialize

func (a *SpurAnnotatorFactory) Initialize(_ *GlobalConf) error

func (*SpurAnnotatorFactory) IsEnabled

func (a *SpurAnnotatorFactory) IsEnabled() bool

func (*SpurAnnotatorFactory) MakeAnnotator

func (a *SpurAnnotatorFactory) MakeAnnotator(i int) Annotator

Directories

Path Synopsis
cmd
mrt2json command
zannotate command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL