ffs

package module
v0.4.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2022 License: Apache-2.0 Imports: 13 Imported by: 3

README

crashplan-ffs-go-pkg

A third-party Golang package for Code42's Crashplan Forensic File Search (FFS) API

The goal of this Golang package is to provide an easy to use/integrate package for Code42's Crashplan FFS API within the Golang environment. There are two main functions that can be used within the package:

  1. GetAuthData
  2. GetFileEvents

These functions allow for someone to query the Crashplan FFS API and get the results returned in a Golang struct which can then be used for other purposes.

GetAuthData function

The GetAuthData is intended to get an API token for a user that will last for one (1) hour, which can then be used with the GetFileEvents function.

Arguments:

  • uri - This is the URL which will provide the API token. (I believe it will always be: https://www.crashplan.com/c42api/v3/auth/jwt?useBody=true)
  • username - The username of the account that has permissions to access the FFS API. (This must be an email address according to the API)
  • password - The password of the account that is set as the username.

Returns:

  • AuthData - Golang struct that contains the API token.
#AuthData struct structure
AuthData
    Data            AuthToken

AuthToken
    V3UserToken     string
  • error - Any errors.

GetFileEvents function

The GetFileEvents is intended to gather all events for a passed query and return them as a Golang struct slice.

Arguments:

  • authData - This is the Golang struct which is gotten from the GetAuthData function.
  • ffsURI - This is the URL which actually hosts the FFS API. (See Code42 documentation for URI, default is https://forensicsearch-default.prod.ffs.us2.code42.com/forensic-search/queryservice/api/v1/)
  • query - This is the properly formatted FFS Query struct which is what is actually executed against the Code42 Crashplan FFS API. (See documentation for how to properly format these queries.)
    • Example JSON query (Returns all events within a 5 second delta)
#Json Format
{
    "groups":[
       {
          "filters":[
             {
                "operator":"IS",
                "term":"fileName",
                "value":"*"
             },
             {
                "operator":"ON_OR_AFTER",
                "term":"insertionTimestamp",
                "value":"2019-08-18T20:31:48.728Z"
             },
             {
                "operator":"ON_OR_BEFORE",
                "term":"insertionTimestamp",
                "value":"2019-08-18T20:32:03.728Z"
             }
          ],
          "filterClause":"AND"
       }
    ],
    "groupClause":"AND",
    "pgNum":1,
    "pgSize":100,
    "srtDir":"asc",
    "srtKey":"insertionTimestamp"
}

#Query Struct format
Query
	Groups 		    []Group
	GroupClause     string      (optional)
	PgNum 		    int         (optional)
	PgSize 		    int         (optional)
	SrtDir 		    string      (optional)
	SrtKey 		    string      (optional)
}

Group
	Filters 	    []Filter
    FilterClause 	string      (optional)
}

Filter
	Operator 	    string
	Term 		    string
	Value 		    string
}

Returns:

  • []FileEvent - Golang struct slice that contains all events returned from the jsonQuery string
#FileEvent struct structure
FileEvent
    EventId                     string
    EventType                   string
    EventTimestamp              *time.Time      (potentially empty)
    InsertionTimestamp          *time.Time      (potentially empty)
    FilePath                    string          (potentially empty)
    FileName                    string
    FileType                    string          (potentially empty)
    FileCategory                string          (potentially empty)
    IdentifiedExtensionCategory string          (potentially empty)
    CurrentExtensionCategory    string          (potentially empty)
    FileSize                    *int            (potentially empty)
    FileOwner                   []string        (potentially empty)
    Md5Checksum                 string	        (potentially empty)
    Sha256Checksum              string	        (potentially empty)
    CreatedTimestamp            *time.Time      (potentially empty)
    ModifyTimestamp             *time.Time      (potentially empty)
    DeviceUsername              string          (potentially empty)
    DeviceUid                   string          (potentially empty)
    UserUid                     string          (potentially empty)
    OsHostname                  string          (potentially empty)
    DomainName                  string          (potentially empty)
    PublicIpAddress             string	        (potentially empty)
    PrivateIpAddresses          []string        (potentially empty)
    Actor                       string	        (potentially empty)
    DirectoryId                 []string        (potentially empty)
    Source                      string          (potentially empty)
    Url                         string	        (potentially empty)
    Shared                      *bool	        (potentially empty)
    SharedWith                  []string        (potentially empty)
    SharingTypeAdded            []string        (potentially empty)
    CloudDriveId                string	        (potentially empty)
    DetectionSourceAlias        string	        (potentially empty)
    FileId                      string	        (potentially empty)
    Exposure                    []string        (potentially empty)
    ProcessOwner                string	        (potentially empty)
    ProcessName                 string	        (potentially empty)
    TabWindowTitle              string          (potentially empty)
    TabUrl                      string          (potentially empty)
    RemovableMediaVendor        string	        (potentially empty)
    RemovableMediaName          string	        (potentially empty)
    RemovableMediaSerialNumber  string	        (potentially empty)
    RemovableMediaCapacity      *int            (potentially empty)
    RemovableMediaBusType       string	        (potentially empty)
    RemovableMediaMediaName     string          (potentially empty)
    RemovableMediaVolumeName    string          (potentially empty)
    RemovableMediaPartitionId   string          (potentially empty)
    SyncDestination             string	        (potentially empty)
    SyncDestinationUsername     string          (potentially empty)
    EmailDLPPolicyNames         []string        (potentially empty)
    EmailDLPSubject             string          (potentially empty)
    EmailDLPSender              string          (potentially empty)
    EmailDLPFrom                string          (potentially empty)
    EmailDLPRecipients          []string        (potentially empty)
    OutsideActiveHours          *bool           (potentially empty)
    IdentifiedExtensionMIMEType string          (potentially empty)
    CurrentExtensionMIMEType    string          (potentially empty)
    SuspiciousFileTypeMismatch  *bool           (potentially empty)
    PrintJobName                string          (potentially empty)
    PrinterName                 string          (potentially empty)
    PrintedFilesBackupPath      string          (potentially empty)
    RemoteActivity              string          (potentially empty)
    Trusted                     *bool           (potentially empty)
    LoggedInOperatingSystemUser string          (potentially empty)
  • error - Any errors.

Limitations:

Code42 Crashplan FFS API has limitations like most APIs, these limitations affect the GetFileEvents function:

  1. 120 Queries per minute, any additional queries will be dropped. (never actually bothered to test if/how this limit is actually enforced)
  2. 200,000 results returned per query. This limitation is kind of annoying to handle as there is no easy way to handle it. The API does not support paging and the only way to figure out how many results there is for a query is to first query, count, then if over 200,000 results, break up the query into smaller time increments and perform multiple queries to get all the results.
  3. The GetFileEvents function only supports the /v1/fileevent/export API endpoint currently. This has to do with how the highly limited functionality of the /v1/fileevent endpoint which isn't well documented.

Code42 Documentation

Links for Code42 Documentation

TODOs

  1. Figure out a way to build tests for these functions

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetCsvFileEvents added in v0.4.1 

func GetCsvFileEvents(authData AuthData, ffsURI string, query Query) (*[]CsvFileEvent, error)

getCsvFileEvents - Function to get the actual event records from FFS *http.Response from ExecQuery This function contains a panic if the csv columns do not match the currently specified list. This is to prevent data from being messed up during parsing.

func GetJsonFileEvents added in v0.4.1 

func GetJsonFileEvents(authData AuthData, ffsURI string, query Query, pgToken string, debugging bool) (*[]JsonFileEvent, string, error)

Types

type AuthData 

type AuthData struct {
	AccessToken string `json:"access_token"`
	Error       string `json:"error,omitempty"`
	Warnings    string `json:"warnings,omitempty"`
	TokenType   string `json:"token_type,omitempty"`
	ExpiresIn   *int   `json:"expires_in,omitempty"`
}

AuthData Structs of Crashplan FFS API Authentication Token Return

func GetAuthData 

func GetAuthData(uri string, username string, password string) (*AuthData, error)

GetAuthData - Function to get the Authentication data (mainly the authentication token) which will be needed for the rest of the API calls The authentication token is good for up to 1 hour before it expires

type CsvFileEvent added in v0.4.1 

type CsvFileEvent struct {
	EventId                     string     `json:"eventId,omitempty"`
	EventType                   string     `json:"eventType,omitempty"`
	EventTimestamp              *time.Time `json:"eventTimestamp,omitempty"`
	InsertionTimestamp          *time.Time `json:"insertionTimestamp,omitempty"`
	FilePath                    string     `json:"filePath,omitempty"`
	FileName                    string     `json:"fileName,omitempty"`
	FileType                    string     `json:"fileType,omitempty"`
	FileCategory                string     `json:"fileCategory,omitempty"`
	IdentifiedExtensionCategory string     `json:"identifiedExtensionCategory,omitempty"`
	CurrentExtensionCategory    string     `json:"currentExtensionCategory,omitempty"`
	FileSize                    *int       `json:"fileSize,omitempty"`
	FileOwner                   []string   `json:"fileOwner,omitempty"` //Array of owners
	Md5Checksum                 string     `json:"md5Checksum,omitempty"`
	Sha256Checksum              string     `json:"sha256Checksum,omitempty"`
	CreatedTimestamp            *time.Time `json:"createdTimestamp,omitempty"`
	ModifyTimestamp             *time.Time `json:"modifyTimestamp,omitempty"`
	DeviceUsername              string     `json:"deviceUsername,omitempty"`
	DeviceUid                   string     `json:"deviceUid,omitempty"`
	UserUid                     string     `json:"userUid,omitempty"`
	OsHostname                  string     `json:"osHostname,omitempty"`
	DomainName                  string     `json:"domainName,omitempty"`
	PublicIpAddress             string     `json:"publicIpAddress,omitempty"`
	PrivateIpAddresses          []string   `json:"privateIpAddresses,omitempty"` //Array of IP address strings
	Actor                       string     `json:"actor,omitempty"`
	DirectoryId                 []string   `json:"directoryId,omitempty"` //An array of something, I am not sure
	Source                      string     `json:"source,omitempty"`
	Url                         string     `json:"url,omitempty"`
	Shared                      *bool      `json:"shared,omitempty"`
	SharedWith                  []string   `json:"sharedWith,omitempty"` //An array of strings (Mainly Email Addresses)
	SharingTypeAdded            []string   `json:"sharingTypeAdded,omitempty"`
	CloudDriveId                string     `json:"cloudDriveId,omitempty"`
	DetectionSourceAlias        string     `json:"detectionSourceAlias,omitempty"`
	FileId                      string     `json:"fileId,omitempty"`
	Exposure                    []string   `json:"exposure,omitempty"`
	ProcessOwner                string     `json:"processOwner,omitempty"`
	ProcessName                 string     `json:"processName,omitempty"`
	TabWindowTitle              string     `json:"tabWindowTitle,omitempty"`
	TabUrl                      string     `json:"tabUrl,omitempty"`
	TabTitles                   []string   `json:"tabTitles,omitempty"`
	TabURLs                     []string   `json:"tabURLs,omitempty"`
	RemovableMediaVendor        string     `json:"removableMediaVendor,omitempty"`
	RemovableMediaName          string     `json:"removableMediaName,omitempty"`
	RemovableMediaSerialNumber  string     `json:"removableMediaSerialNumber,omitempty"`
	RemovableMediaCapacity      *int       `json:"removableMediaCapacity,omitempty"`
	RemovableMediaBusType       string     `json:"removableMediaBusType,omitempty"`
	RemovableMediaMediaName     string     `json:"removableMediaMediaName,omitempty"`
	RemovableMediaVolumeName    string     `json:"removableMediaVolumeName,omitempty"`
	RemovableMediaPartitionId   string     `json:"removableMediaPartitionId,omitempty"`
	SyncDestination             string     `json:"syncDestination,omitempty"`
	SyncDestinationUsername     string     `json:"syncDestinationUsername,omitempty"`
	EmailDLPPolicyNames         []string   `json:"emailDLPPolicyNames,omitempty"`
	EmailDLPSubject             string     `json:"emailDLPSubject,omitempty"`
	EmailDLPSender              string     `json:"emailDLPSender,omitempty"`
	EmailDLPFrom                string     `json:"emailDLPFrom,omitempty"`
	EmailDLPRecipients          []string   `json:"emailDLPRecipients,omitempty"`
	OutsideActiveHours          *bool      `json:"outsideActiveHours,omitempty"`
	IdentifiedExtensionMIMEType string     `json:"identifiedExtensionMimeType,omitempty"`
	CurrentExtensionMIMEType    string     `json:"currentExtensionMimeType,omitempty"`
	SuspiciousFileTypeMismatch  *bool      `json:"suspiciousFileTypeMismatch,omitempty"`
	PrintJobName                string     `json:"printJobName,omitempty"`
	PrinterName                 string     `json:"printerName,omitempty"`
	PrintedFilesBackupPath      string     `json:"printedFilesBackupPath,omitempty"`
	RemoteActivity              string     `json:"remoteActivity,omitempty"`
	Trusted                     *bool      `json:"trusted,omitempty"`
	LoggedInOperatingSystemUser string     `json:"loggedInOperatingSystemUser,omitempty"`
	DestinationCategory         string     `json:"destinationCategory,omitempty"`
	DestinationName             string     `json:"destinationName,omitempty"`
}

The CSV main body of a file event record

type FieldError added in v0.4.1 

type FieldError struct {
	Error string `json:"error,omitempty"`
	Field string `json:"field,omitempty"`
}

type Group 

type Group struct {
	Filters      []SearchFilter `json:"filters"`
	FilterClause string         `json:"filterClause,omitempty"`
}

type JsonFileEvent added in v0.4.1 

type JsonFileEvent struct {
	Actor                      string       `json:"actor,omitempty"`
	CloudDriveId               string       `json:"cloudDriveId,omitempty"`
	CreateTimestamp            string       `json:"createTimestamp,omitempty"`
	DestinationCategory        string       `json:"destinationCategory,omitempty"`
	DestinationName            string       `json:"destinationName,omitempty"`
	DetectionSourceAlias       string       `json:"detectionSourceAlias,omitempty"`
	DeviceUid                  string       `json:"deviceUid,omitempty"`
	DeviceUserName             string       `json:"deviceUserName,omitempty"`
	DirectoryId                []string     `json:"directoryId,omitempty"`
	DomainName                 string       `json:"domainName,omitempty"`
	EmailDlpPolicyNames        []string     `json:"emailDlpPolicyNames,omitempty"`
	EmailFrom                  string       `json:"emailFrom,omitempty"`
	EmailRecipients            []string     `json:"emailRecipients,omitempty"`
	EmailSender                string       `json:"emailSender,omitempty"`
	EmailSubject               string       `json:"emailSubject,omitempty"`
	EventId                    string       `json:"eventId"`
	EventTimestamp             string       `json:"eventTimestamp,omitempty"`
	EventType                  string       `json:"eventType,omitempty"`
	Exposure                   []string     `json:"exposure,omitempty"`
	FieldErrors                []FieldError `json:"fieldErrors,omitempty"`
	FileCategory               string       `json:"fileCategory,omitempty"`
	FileCategoryByBytes        string       `json:"fileCategoryByBytes,omitempty"`
	FileCategoryByExtension    string       `json:"fileCategoryByExtension,omitempty"`
	FileId                     string       `json:"fileId,omitempty"`
	FileName                   string       `json:"fileName,omitempty"`
	FileOwner                  string       `json:"fileOwner,omitempty"`
	FilePath                   string       `json:"filePath,omitempty"`
	FileSize                   *int64       `json:"fileSize,omitempty"`
	FileType                   string       `json:"fileType,omitempty"`
	InsertionTimestamp         string       `json:"insertionTimestamp,omitempty"`
	Md5Checksum                string       `json:"md5Checksum,omitempty"`
	MimeTypeByBytes            string       `json:"mimeTypeByBytes,omitempty"`
	MimeTypeByExtension        string       `json:"mimeTypeByExtension,omitempty"`
	MimeTypeMismatch           *bool        `json:"mimeTypeMismatch,omitempty"`
	ModifyTimestamp            string       `json:"modifyTimestamp,omitempty"`
	OperatingSystemUser        string       `json:"operatingSystemUser,omitempty"`
	OsHostName                 string       `json:"osHostName,omitempty"`
	OutsideActiveHours         *bool        `json:"outsideActiveHours,omitempty"`
	PrintJobName               string       `json:"printJobName,omitempty"`
	PrinterName                string       `json:"printerName,omitempty"`
	PrivateIpAddresses         []string     `json:"privateIpAddresses,omitempty"`
	ProcessName                string       `json:"processName,omitempty"`
	ProcessOwner               string       `json:"processOwner,omitempty"`
	PublicIpAddress            string       `json:"publicIpAddress,omitempty"`
	RemoteActivity             string       `json:"remoteActivity,omitempty"`
	RemovableMediaBusType      string       `json:"removableMediaBusType,omitempty"`
	RemovableMediaCapacity     *int64       `json:"removableMediaCapacity,omitempty"`
	RemovableMediaMediaName    string       `json:"removableMediaMediaName,omitempty"`
	RemovableMediaName         string       `json:"removableMediaName,omitempty"`
	RemovableMediaPartitionId  []string     `json:"removableMediaPartitionId,omitempty"`
	RemovableMediaSerialNumber string       `json:"removableMediaSerialNumber,omitempty"`
	RemovableMediaVendor       string       `json:"removableMediaVendor,omitempty"`
	RemovableMediaVolumeName   []string     `json:"removableMediaVolumeName,omitempty"`
	Sha256Checksum             string       `json:"sha256Checksum,omitempty"`
	Shared                     string       `json:"shared,omitempty"`
	SharedWith                 []SharedWith `json:"sharedWith,omitempty"`
	SharingTypeAdded           []string     `json:"sharingTypeAdded,omitempty"`
	Source                     string       `json:"source,omitempty"`
	SyncDestination            string       `json:"syncDestination,omitempty"`
	SyncDestinationUsername    []string     `json:"syncDestinationUsername,omitempty"`
	TabUrl                     string       `json:"tabUrl,omitempty"`
	Tabs                       []Tab        `json:"tabs,omitempty"`
	Trusted                    *bool        `json:"trusted,omitempty"`
	Url                        string       `json:"url,omitempty"`
	UserUid                    string       `json:"userUid,omitempty"`
	WindowTitle                []string     `json:"windowTitle,omitempty"`
}

type JsonFileEventResponse added in v0.4.1 

type JsonFileEventResponse struct {
	FileEvents  []JsonFileEvent `json:"fileEvents,omitempty"`
	NextPgToken string          `json:"nextPgToken,omitempty"`
	Problems    []QueryProblem  `json:"problems,omitempty"`
	TotalCount  *int64          `json:"totalCount,omitempty"`
}

func GetJsonFileEventResponse added in v0.4.1 

func GetJsonFileEventResponse(resp *http.Response) (*JsonFileEventResponse, error)

type Query 

type Query struct {
	Groups      []Group `json:"groups"`
	GroupClause string  `json:"groupClause,omitempty"`
	PgNum       int     `json:"pgNum,omitempty"`
	PgSize      int     `json:"pgSize,omitempty"`
	PgToken     string  `json:"pgToken"`
	SrtDir      string  `json:"srtDir,omitempty"`
	SrtKey      string  `json:"srtKey,omitempty"`
}

Structs for FFS Queries

type QueryProblem added in v0.4.1 

type QueryProblem struct {
	BadFilter   SearchFilter `json:"badFilter,omitempty"`
	Description string       `json:"description,omitempty"`
	Type        string       `json:"type,omitempty"`
}

type SearchFilter added in v0.4.1 

type SearchFilter struct {
	Operator string `json:"operator"`
	Term     string `json:"term"`
	Value    string `json:"value"`
}

type SharedWith added in v0.4.1 

type SharedWith struct {
	CloudUsername *string `json:"cloudUsername,omitempty"`
}

type Tab added in v0.4.1 

type Tab struct {
	Title string `json:"title,omitempty"`
	Url   string `json:"url,omitempty"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.