README
¶
Helios
Helios is a GUI automation tool written in Go - it automates anything that you can see on your screen. Helios uses image recognition powered by OpenCV under-the-hood. You can use Helios to automate GUI testing, video games and more.
Example:
package main
import (
. "github.com/j7mbo/helios"
"time"
)
func main() {
// Initialise helios without any configuration
app := NewApplication(nil)
// Load .gopher.png and expect to find it at 0.9 / 1.0 confidence
gopherImage, _ := NewImage("./gopher.png", 0.9)
// Find the gopher image on-screen
gopher := app.GetScreen().Find(gopherImage)
// Highlight it for two seconds
gopher.Highlight(2 * time.Second)
// Click at a random (X,Y) coordinate within the matched box
gopher.Click()
}
The Gopher from Goland, highlighted for 2 seconds:
Installation
MacOS
brew install pkg-config
brew install opencv
Docs
Those using a mac with a retina display
When you take a screenshot of something you want to search for on MacOS, like the gopher icon, (use command, control, shift, 4), you need to adjust the image size (if you're using a retina display). To do this, open the image in preview, go to Tools > Adjust Size and then choose 200 per cent for the width and height. Then save. Your image should now be findable.
Upcoming work
- Ability to define
Regions to search - Abstraction for
finder - Works with any resolution
- Wait X seconds for an image to appear
- Offsets for regions
- Stable supported api
- Support and releases for Windows, Linux (MacOS already supported)
- Moving highlight box during highlighting
- Support for multiple screens
- Fix for issue #1 - no need to embed a highlighter binary
- Embedding OpenCV directly, 0 dependency binary (need help with this)
- API for interfacing with other languages
- OCR, text extraction
- GUI for region and image selection, scripting language (this'll be a fun one)
- Tests...
Documentation
¶
Index ¶
- Variables
- type App
- type Clicker
- type Config
- type Finder
- type HighlightRequest
- type Highlighter
- type Image
- type Match
- type Point
- type PollInterval
- type Region
- func (r *Region) Click()
- func (r *Region) Find(i *Image) *Match
- func (r *Region) FindAll(i *Image) []*Match
- func (r *Region) GetHeight() int
- func (r *Region) GetTopLeft() *Point
- func (r *Region) GetWidth() int
- func (r *Region) Highlight(t time.Duration)
- func (r *Region) MoveMouse()
- func (r *Region) Offset(x, y, width, height int) *Region
- func (r *Region) Wait(i *Image, t time.Duration) *Match
- type Screen
Constants ¶
This section is empty.
Variables ¶
var ImageNotFoundError = errors.New("file not found")
var ImageUnknownError = errors.New("unknown error")
var ImageUnsupportedFormatError = errors.New("unsupported format")
var InvalidConfigurationError = errors.New("invalid configuration provided")
Functions ¶
This section is empty.
Types ¶
type Clicker ¶
type Clicker struct {
}
func (*Clicker) Click ¶
Click clicks at a random X and Y coordinate within the given location. This doesn't use the standard Click() function of any library which is easy to detect as an automated click - rather it adds entropy throughout the mouse events.
func (*Clicker) MoveMouseInRegion ¶
type Config ¶
type Config struct {
// contains filtered or unexported fields
}
func (*Config) GetPollInterval ¶
func (c *Config) GetPollInterval() *PollInterval
type Finder ¶
type Finder struct {
// contains filtered or unexported fields
}
func NewFinder ¶
func NewFinder(screen *Screen, pollInterval *PollInterval) *Finder
type HighlightRequest ¶
type HighlightRequest struct {
ScreenWidth int `arg:"--screen-width"`
ScreenHeight int `arg:"--screen-height"`
X float64 `arg:"--x"`
Y float64 `arg:"--y"`
Width float64 `arg:"--w"`
Height float64 `arg:"--h"`
Duration float64 `arg:"--d"`
}
func (HighlightRequest) AsArgsArray ¶
func (r HighlightRequest) AsArgsArray() []string
type Highlighter ¶
type Highlighter struct {
// contains filtered or unexported fields
}
func NewHighlighter ¶
func NewHighlighter(highlighterBinary []byte) *Highlighter
func (*Highlighter) Highlight ¶
func (h *Highlighter) Highlight(r *HighlightRequest)
type Match ¶
type Match struct {
Region
// contains filtered or unexported fields
}
func (*Match) GetConfidence ¶
type PollInterval ¶
type Region ¶
type Region struct {
// contains filtered or unexported fields
}
func (*Region) GetTopLeft ¶
type Screen ¶
type Screen struct {
Region
// contains filtered or unexported fields
}
func NewScreen ¶
func NewScreen(highlighter *Highlighter, interval *PollInterval) *Screen
Source Files
Directories