README
¶
ripe-atlas
- RIPE Atlas v2 API access in Go. *
ripe-atlas is a Go library to access the RIPE Atlas REST API.
It features a simple CLI-based tool called atlas which serve both as a collection of use-cases for the library and an easy way to use it.
Work in progress, still incomplete
Table of content
Features
I am trying to implement the full REST API in Go. The API itself is not particularly complex but the settings and parameters are.
The following topic are available:
-
probes
you can query one probe or ask for a list of probes with a few criterias
-
measurements
you can create and list measurements.
-
results
every measurement has a URI in the result json that points to the actual results. This fetch and display them.
In addition to these major commands, there are a few shortcut commands (see below):
- dns
- http
- ip
- keys
- ntp
- ping
- sslcert/tls
- traceroute
Installation
NOTE: you MUST have Go 1.8 or later. Previous versions did not have the ProxyHeader fields and thus no support for HTTP proxy.
Like many Go-based tools, installation is very easy
go get github.com/keltia/ripe-atlas/cmd/...
or
git clone https://github.com/keltia/ripe-atlas
make install
The library is fetched, compiled and installed in whichever directory is specified by $GOPATH. The atlas binary will also be installed (on windows, this will be called atlas.exe).
You can install the dependencies with go get
github.com/urfave/cligithub.com/naoina/toml
To run the tests, you will also need:
github.com/stretchr/assert
NOTE: please use and test the Windows version (use make windowsto generate it). It should work but I lack resources to play much with it.
API usage
You must foremost instanciate a new API client with
client, err := atlas.NewClient(config)
where config is an atlas.Config{} struct with various options.
All API calls after that will use client:
probe, err := client.GetProbe(12345)
Basics
- Authentication
- Probes
- Measurements
- Applications
CLI utility
The atlas command is a command-line client for the Go API.
Configuration
The atlas utility uses a configuration file in the TOML file format.
On UNIX, it is located in $HOME/.config/ripe-atlas/config.toml and in %LOCALAPPDATA%\RIPE-ATLAS on Windows.
There are only a few parameters for now, the most important one being your API Key for authenticate against the RIPE API endpoint. You can now specify the default probe set (and override it from the CLI):
# Default configuration file
API_key = "<INSERT-API-KEY>"
default_probe = <PROBE-ID>
[probe_set]
pool_size = <POOL-SIZE>
type = "area"
value = "WW"
Everything is a string except for pool_size and default_probe which are integers.
Be aware that if you ask for an IPv6 object (like a domain or machine name), the API will refuse your request if the IPv6 version of that object does not exist.
Important note
Not all parameters specified for the different commands are implemented, as you can see in the API Reference, there are a lot of different parameters like all the id__{gt,gte,lt,lte,in} stuff.
Proxy authentication
If you want to use atlas or the API behind an authenticating HTTP/HTTPS proxy, you need to create another configuration file holding the proxy credentials. The file is called .netrc under UNIX and is located in your $HOME directory. On Windows, it is in the same directory as config.toml inside %LOCALAPPDATA%\RIPE-ATLAS.
The format is described in the ftp(1) manpage:
machine <service> username <username> password <password>
in our case, the service is proxy (or default), you just need to fill in username and password.
machine proxy username john.doe password secret
The API now look by itself for the .netrc file, using my own github.com/keltia/proxy package.
atlas.go:
import "github.com/keltia/proxy"
// Check whether we have proxy authentication (from a separate config file)
authstr, err := proxy.SetupProxyAuth()
client, err = atlas.NewClient(atlas.Config{
APIKey: mycnf.APIKey,
DefaultProbe: mycnf.DefaultProbe,
PoolSize: mycnf.PoolSize,
ProxyAuth: auth,
Verbose: fVerbose,
})
As an alternative, you can do the encoding yourself and put that in config.toml as proxy_auth.
Last, you can also use the full form for the https_proxy environment variable with user:password@proxy but it is not recommended to put your password in the clear like this.
Usage
NAME:
atlas - RIPE Atlas CLI interface
USAGE:
atlas [global options] command [command options] [arguments...]
VERSION:
0.41
AUTHOR:
Ollivier Robert <roberto@keltia.net>
COMMANDS:
credits, c credits-related keywords
dns, dig, drill send dns queries
http, https connect to host/IP through HTTP
ip returns current ip
keys, k, key key-related keywords
measurements, measures, m measurements-related keywords
ntp get time from ntp server
ping ping selected address
probes, p, pb probe-related keywords
results, r, res results for one measurement
sslcert, tlscert, tls get TLS certificate from host/IP
traceroute, trace traceroute to given host/IP
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--format value, -f value specify output format (NOT IMPLEMENTED)
--debug, -D debug mode
--verbose, -v verbose mode
--fields value, -F value specify which fields are wanted
--include value, -I value specify whether objects should be expanded
--logfile value, -L value specify a log file
--mine, -M limit output to my objects
--opt-fields value, -O value specify which optional fields are wanted
--page-size value, -P value page size for results
--sort value, -S value sort results
-1, --is-oneoff one-time measurement
-6, --ipv6 Only IPv6
-4, --ipv4 Only IPv4
--pool-size value, -N value Number of probes to request (default: 0)
--area-type value Set type for probes (area, country, etc.)
--area-value value Value for the probe set (WW, West, etc.)
--country value, -C value Short cut to specify a country
--tags value, -T value Include/exclude tags for probesets
--help, -h show help
--version, -V
In addition to the main probes and measurements commands, it features fast-access to common tasks like pingand traceroute.
Now, every command has a -T or -tags parameter to add user-defined tags when the measurement is created. This is different from -T at the higher level. The latter is for selecting probes.
$ atlas ping -h
NAME:
atlas ping - ping selected address
USAGE:
atlas ping [command options] [arguments...]
DESCRIPTION:
send echo/reply to an IP
OPTIONS:
-T value, --tags value add tags to measurement
You can use it like that:
$ atlas -N 10 -4 ping -T test-tag,foobar www.example.com
When looking at measurement results, it is very easy to use something like jq to properly display JSON data:
atlas results <ID> | jq .
You can also analyze the results, as explained here.
Here,to find the maximum RTT:
% ./atlas measurements results 10185594 | jq 'map(.result[0].rtt) | max'
24.10811
And with this jq file, to get more information from a measurement:
% cat ping-report.jq
map(.result) | flatten(1) | map(.rtt) | length as $total |
"Median: " + (sort |
if length % 2 == 0 then .[length/2] else .[(length-1)/2] end | tostring),
"Average: " + (map(select(. != null)) | add/length | tostring) + " ms",
"Min: " + (map(select(. != null)) | min | tostring) + " ms",
"Max: " + (max | tostring) + " ms",
"Failures: " + (map(select(. == null)) | (length*100/$total) | tostring) + " %"
%./atlas measurements results 10185594 | jq --raw-output --from-file ping-report.jq
Median: 15.068505
Average: 15.480822916666666 ms
Min: 3.786365 ms
Max: 24.164375 ms
Failures: 14.285714285714286 %
TODO
- implement "anchors" & "participation requests"
- more tests (and better ones!)
- better display of results
- refactoring to reduce code duplication: always in progress
- even more tests
External Documentation
Contributing
Please see CONTRIBUTING.md for some simple rules.
Documentation
¶
Index ¶
- Variables
- func AddQueryParameters(baseURL string, queryParams map[string]string) string
- func FillDefinition(d *Definition, fields map[string]string) error
- func GetVersion() string
- type APIError
- type Client
- func (c *Client) DNS(d *MeasurementRequest) (m *MeasurementResp, err error)
- func (c *Client) DeleteMeasurement(id int) (err error)
- func (c *Client) FetchResult(url string) (string, error)
- func (c *Client) GetCredits() (credits *Credits, err error)
- func (c *Client) GetKey(uuid string) (k Key, err error)
- func (c *Client) GetKeys(opts map[string]string) (kl []Key, err error)
- func (c *Client) GetMeasurement(id int) (m *Measurement, err error)
- func (c *Client) GetMeasurements(opts map[string]string) (m []Measurement, err error)
- func (c *Client) GetProbe(id int) (p *Probe, err error)
- func (c *Client) GetProbes(opts map[string]string) (p []Probe, err error)
- func (c *Client) HTTP(d *MeasurementRequest) (m *MeasurementResp, err error)
- func (c *Client) HasAPIKey() (string, bool)
- func (c *Client) NTP(d *MeasurementRequest) (m *MeasurementResp, err error)
- func (c *Client) NewMeasurement() (req *MeasurementRequest)
- func (c *Client) Ping(d *MeasurementRequest) (m *MeasurementResp, err error)
- func (c *Client) SSLCert(d *MeasurementRequest) (m *MeasurementResp, err error)
- func (c *Client) SetOption(name, value string) *Client
- func (c *Client) Traceroute(d *MeasurementRequest) (m *MeasurementResp, err error)
- type Config
- type Credits
- type Definition
- type Grant
- type Key
- type Measurement
- type MeasurementRequest
- type MeasurementResp
- type ParticipationRequest
- type Probe
- type ProbeSet
Constants ¶
This section is empty.
Variables ¶
var ( // ProbeTypes should be obvious ProbeTypes = []string{"area", "country", "prefix", "asn", "probes", "msm"} // AreaTypes should also be obvious AreaTypes = []string{"WW", "West", "North-Central", "South-Central", "North-East", "South-East"} )
var ErrAPIKeyIsMandatory = errors.New("API call requires an API key")
ErrAPIKeyIsMandatory is returned when a call need one
var ErrInvalidAPIKey = errors.New("invalid API key")
ErrInvalidAPIKey is returned when the key is invalid
var ErrInvalidError = errors.New("Invalid request")
ErrInvalidAPIAnswer
var ErrInvalidMeasurementType = errors.New("invalid measurement type")
ErrInvalidMeasurementType is a new error
Functions ¶
func AddQueryParameters ¶
AddQueryParameters adds query parameters to the URL.
func FillDefinition ¶
func FillDefinition(d *Definition, fields map[string]string) error
FillDefinition set a few parameters in a definition list
The goal here is to give a dictionary of string and let it figure out each field's type depending on the recipient's type in the struct.
Types ¶
type APIError ¶
type APIError struct {
Err struct {
Status int `json:"status"`
Code int `json:"code"`
Detail string `json:"detail"`
Title string `json:"title"`
Errors []struct {
Source struct {
Pointer string
} `json:"source"`
Detail string
} `json:"errors"`
} `json:"error"`
}
APIError is for errors returned by the RIPE API.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is the main struct holding state in an API client
func NewClient ¶
NewClient is the first function to call. Yes, it does take multiple config and the last one wins.
func (*Client) DNS ¶
func (c *Client) DNS(d *MeasurementRequest) (m *MeasurementResp, err error)
DNS creates a measurement
func (*Client) DeleteMeasurement ¶
DeleteMeasurement stops (not really deletes) a given measurement
func (*Client) FetchResult ¶
FetchResult downloads result for a given measurement
func (*Client) GetCredits ¶
GetCredits returns high-level data for credits
func (*Client) GetMeasurement ¶
func (c *Client) GetMeasurement(id int) (m *Measurement, err error)
GetMeasurement gets info for a single one
func (*Client) GetMeasurements ¶
func (c *Client) GetMeasurements(opts map[string]string) (m []Measurement, err error)
GetMeasurements gets info for a set
func (*Client) HTTP ¶
func (c *Client) HTTP(d *MeasurementRequest) (m *MeasurementResp, err error)
HTTP creates a measurement
func (*Client) NTP ¶
func (c *Client) NTP(d *MeasurementRequest) (m *MeasurementResp, err error)
NTP creates a measurement
func (*Client) NewMeasurement ¶
func (c *Client) NewMeasurement() (req *MeasurementRequest)
NewMeasurement create a new MeasurementRequest and fills some fields
func (*Client) Ping ¶
func (c *Client) Ping(d *MeasurementRequest) (m *MeasurementResp, err error)
Ping creates a measurement
func (*Client) SSLCert ¶
func (c *Client) SSLCert(d *MeasurementRequest) (m *MeasurementResp, err error)
SSLCert creates a measurement
func (*Client) Traceroute ¶
func (c *Client) Traceroute(d *MeasurementRequest) (m *MeasurementResp, err error)
Traceroute creates a measurement
type Config ¶
type Config struct {
APIKey string
DefaultProbe int
AreaType string
AreaValue string
IsOneOff bool
PoolSize int
WantAF string
ProxyAuth string
Verbose bool
Tags string
Log *log.Logger
Level int
// contains filtered or unexported fields
}
Config is the main object when creating an API Client
type Credits ¶
type Credits struct {
CurrentBalance int `json:"current_balance"`
EstimatedDailyIncome int `json:"estimated_daily_income"`
EstimatedDailyExpenditure int `json:"estimated_daily_expenditure"`
EstimatedDailyBalance int `json:"estimated_daily_balance"`
CalculationTime string `json:"calculation_time"`
EstimatedRunoutSeconds int `json:"estimated_runout_seconds"`
PastDayMeasurementResults int `json:"past_day_measurement_results"`
PastDayCreditsSpent int `json:"past_day_credits_spent"`
IncomeItems string `json:"income_items"`
ExpenseItems string `json:"expense_items"`
Transactions string `json:"transactions"`
}
Credits is holding credits data
type Definition ¶
type Definition struct {
// Required fields
Description string `json:"description"`
Type string `json:"type"`
AF int `json:"af"`
// Required for all but "dns"
Target string `json:"target,omitempty"`
GroupID int `json:"group_id,omitempty"`
Group string `json:"group,omitempty"`
InWifiGroup bool `json:"in_wifi_group,omitempty"`
Spread int `json:"spread,omitempty"`
Packets int `json:"packets,omitempty"`
PacketInterval int `json:"packet_interval,omitempty"`
Tags []string `json:"tags,omitempty"`
// Common parameters
ExtraWait int `json:"extra_wait,omitempty"`
IsOneoff bool `json:"is_oneoff,omitempty"`
IsPublic bool `json:"is_public,omitempty"`
ResolveOnProbe bool `json:"resolve_on_probe,omitempty"`
// Default depends on type
Interval int `json:"interval,omitempty"`
// dns & traceroute parameters
Protocol string `json:"protocol,omitempty"`
// dns parameters
QueryClass string `json:"query_class,omitempty"`
QueryType string `json:"query_type,omitempty"`
QueryArgument string `json:"query_argument,omitempty"`
Retry int `json:"retry,omitempty"`
SetCDBit bool `json:"set_cd_bit,omitempty"`
SetDOBit bool `json:"set_do_bit,omitempty"`
SetNSIDBit bool `json:"set_nsid_bit,omitempty"`
SetRDBit bool `json:"set_rd_bit,omitempty"`
UDPPayloadSize int `json:"udp_payload_size,omitempty"`
UseProbeResolver bool `json:"use_probe_resolver"`
// traceroute parameters
DestinationOptionSize int `json:"destination_option_size,omitempty"`
DontFragment bool `json:"dont_fragment,omitempty"`
DuplicateTimeout int `json:"duplicate_timeout,omitempty"`
FirstHop int `json:"first_hop,omitempty"`
HopByHopOptionSize int `json:"hop_by_hop_option_size,omitempty"`
MaxHops int `json:"max_hops,omitempty"`
Paris int `json:"paris,omitempty"`
// http parameters
ExtendedTiming bool `json:"extended_timing,omitempty"`
HeaderBytes int `json:"header_bytes,omitempty"`
Method string `json:"method,omitempty"`
MoreExtendedTiming bool `json:"more_extended_timing,omitempty"`
Path string `json:"path,omitempty"`
QueryOptions string `json:"query_options,omitempty"`
UserAgent string `json:"user_agent,omitempty"`
Version string `json:"version,omitempty"`
// sslcert & traceroute & http parameters
Port int `json:"port,omitempty"`
// ping & traceroute parameters
Size int `json:"size,omitempty"`
// wifi parameters
AnonymousIdentity string `json:"anonymous_identity,omitempty"`
Cert string `json:"cert,omitempty"`
EAP string `json:"eap,omitempty"`
}
Definition is used to create measurements
type Grant ¶
type Grant struct {
Permission string `json:"permission"`
Target struct {
Type string `json:"type"`
ID string `json:"id"`
} `json:"target"`
}
Grant is the permission(s) associated with a key
type Key ¶
type Key struct {
UUID string `json:"uuid"`
ValidFrom string `json:"valid_from"`
ValidTo string `json:"valid_to"`
Enabled bool
IsActive bool `json:"is_active"`
CreatedAt string `json:"created_at"`
Label string `json:"label"`
Grants []Grant `json:"grants"`
Type string `json:"type"`
}
Key is holding the API key parameters
type Measurement ¶
type Measurement struct {
Af int `json:"af"`
CreationTime int `json:"creation_time"`
Description string `json:"description"`
DestinationOptionSize interface{} `json:"destination_option_size"`
DontFragment interface{} `json:"dont_fragment"`
DuplicateTimeout interface{} `json:"duplicate_timeout"`
FirstHop int `json:"first_hop"`
Group string `json:"group"`
GroupID int `json:"group_id"`
HopByHopOptionSize interface{} `json:"hop_by_hop_option_size"`
ID int `json:"id"`
InWifiGroup bool `json:"in_wifi_group"`
Interval int `json:"interval"`
IsAllScheduled bool `json:"is_all_scheduled"`
IsOneoff bool `json:"is_oneoff"`
IsPublic bool `json:"is_public"`
MaxHops int `json:"max_hops"`
PacketInterval interface{} `json:"packet_interval"`
Packets int `json:"packets"`
Paris int `json:"paris"`
ParticipantCount int `json:"participant_count"`
ParticipationRequests []ParticipationRequest `json:"participation_requests"`
Port interface{} `json:"port"`
ProbesRequested int `json:"probes_requested"`
ProbesScheduled int `json:"probes_scheduled"`
Protocol string `json:"protocol"`
ResolveOnProbe bool `json:"resolve_on_probe"`
ResolvedIPs []string `json:"resolved_ips"`
ResponseTimeout int `json:"response_timeout"`
Result string `json:"result"`
Size int `json:"size"`
Spread interface{} `json:"spread"`
StartTime int `json:"start_time"`
Status struct {
ID int `json:"id"`
Name string `json:"name"`
} `json:"status"`
StopTime int `json:"stop_time"`
Target string `json:"target"`
TargetASN int `json:"target_asn"`
TargetIP string `json:"target_ip"`
Type string `json:"type"`
}
Measurement is what we are working with
type MeasurementRequest ¶
type MeasurementRequest struct {
// see below for definition
Definitions []Definition `json:"definitions"`
// requested set of probes
Probes []ProbeSet `json:"probes"`
//
BillTo int `json:"bill_to,omitempty"`
IsOneoff bool `json:"is_oneoff,omitempty"`
SkipDNSCheck bool `json:"skip_dns_check,omitempty"`
Times int `json:"times,omitempty"`
StartTime int `json:"start_time,omitempty"`
StopTime int `json:"stop_time,omitempty"`
}
MeasurementRequest contains the different measurement to create/view
func (*MeasurementRequest) AddDefinition ¶
func (m *MeasurementRequest) AddDefinition(fields map[string]string) *MeasurementRequest
AddDefinition create a new MeasurementRequest and fills some fields
type MeasurementResp ¶
type MeasurementResp struct {
Measurements []int
}
MeasurementResp contains all the results of the measurements
type ParticipationRequest ¶
type ParticipationRequest struct {
Action string `json:"action"`
CreatedAt int `json:"created_at,omitempty"`
ID int `json:"id,omitempty"`
Self string `json:"self,omitempty"`
Measurement string `json:"measurement,omitempty"`
MeasurementID int `json:"measurement_id,omitempty"`
Requested int `json:"requested,omitempty"`
Type string `json:"type,omitempty"`
Value string `json:"value,omitempty"`
Logs string `json:"logs,omitempty"`
}
ParticipationRequest allow you to add or remove probes from a measurement that was already created
type Probe ¶
type Probe struct {
AddressV4 string `json:"address_v4"`
AddressV6 string `json:"address_v6"`
AsnV4 int `json:"asn_v4"`
AsnV6 int `json:"asn_v6"`
CountryCode string `json:"country_code"`
Description string `json:"description"`
FirstConnected int `json:"first_connected"`
Geometry struct {
Type string `json:"type"`
Coordinates []float64 `json:"coordinates"`
} `json:"geometry"`
ID int `json:"id"`
IsAnchor bool `json:"is_anchor"`
IsPublic bool `json:"is_public"`
LastConnected int `json:"last_connected"`
PrefixV4 string `json:"prefix_v4"`
PrefixV6 string `json:"prefix_v6"`
Status struct {
Since string `json:"since"`
ID int `json:"id"`
Name string `json:"name"`
} `json:"status"`
StatusSince int `json:"status_since"`
Tags []struct {
Name string `json:"name"`
Slug string `json:"slug"`
} `json:"tags"`
TotalUptime int `json:"total_uptime"`
Type string `json:"type"`
}
Probe is holding probe's data
type ProbeSet ¶
type ProbeSet struct {
Requested int `json:"requested"` // number of probes
Type string `json:"type"` // area, country, prefix, asn, probes, msm
Value string `json:"value"` // can be numeric or string
TagsInclude string `json:"tags_include,omitempty"`
TagsExclude string `json:"tags_exclude,omitempty"`
}
ProbeSet is a set of probes obviously
Source Files
Directories