infraharvest

infraharvest

infraharvest generates Terraform configuration for infrastructure that already exists. It lists your resources, writes an import block for each, and has Terraform (or OpenTofu) generate the configuration with plan -generate-config-out. It then tidies that configuration and checks that it plans with no changes.

• Clean code, not a dump: references between resources instead of IDs, shared tags as locals, secrets as variables, and clusters of resources as calls of curated modules.
• Checked, not hoped: a change to the generated code is kept only if the plan still shows no changes, every root goes through a verification gate (format, validate, a plan with no changes, scanners, secrets, determinism), and the report says what was imported, what was left out and why.
• Read-only: it never applies anything and writes no state; least-privilege permission templates for each cloud.
• AWS, Azure and Google Cloud, with Terraform or OpenTofu, from the command line, a GitHub Action or an AI agent over MCP.

Terraform's own terraform query and HCP Terraform's Search & Import suit a handful of resources; infraharvest is for whole accounts, subscriptions and projects (see infraharvest and Terraform's own import).

Licensed under AGPL-3.0-only. The documentation is also published at ignatg.github.io/infraharvest.

Documentation

• Quickstart: list, choose, import and plan, on AWS, Azure or Google Cloud.
• Install: releases, the container image, the install script, Go.
• Usage: commands, choosing what to import, services and filters, day-to-day runs, and which options go together.
• Audit: what in an account no Terraform state or other tool manages, without importing anything.
• Configuration: the configuration file and the state backend.
• Output: what each root holds, applying it, how the configuration is tidied, the report and the verification gate.
• Modules: the curated modules clusters of resources move into, your own modules, and generated local modules.
• Permissions: read-only access for AWS, Azure and Google Cloud.
• Troubleshooting: what an error message or exit code means, and what to do.
• GitHub Action: scheduled imports that open a pull request.
• AI agents (MCP) · Catalogue · Command-line reference

Install

On Linux and macOS, including AWS CloudShell, Azure Cloud Shell and Google Cloud Shell once cosign is there (Install shows how):

curl -fsSL https://raw.githubusercontent.com/IgnatG/infraharvest/main/install.sh | sh

The script installs the latest release into ~/.local/bin, after checking its cosign signature and its checksum. The other ways are in Install:

• a release archive for Linux, macOS or Windows, on amd64 or arm64: on Windows, use the zip;
• the container image ghcr.io/ignatg/infraharvest, with Terraform and OpenTofu included;
• go install github.com/IgnatG/infraharvest@latest, with Go 1.27 or later;
• the GitHub Action, in a workflow.

infraharvest needs Terraform 1.5 or later, which it downloads if there is none on PATH, or OpenTofu 1.6 or later with --engine=tofu. It also needs credentials that can read the resources: the permission templates grant read-only access on each cloud. infraharvest version checks the install.

Import

The steps are the same on every cloud:

1. infraharvest discover <provider> lists the resources into selection.yaml, and opens the picker on it in a terminal. By default it leaves out what the cloud creates itself and what another tool manages, with the reason.
2. Review selection.yaml: each resource has include: true or false. infraharvest pick --selection selection.yaml opens the picker again.
3. infraharvest import <provider>, with the same flags and --selection=selection.yaml, writes a root for each account (subscription, project) and region under infrastructure/, and a report in infrastructure/_reports/.

--all in place of --selection imports everything the default selection includes, without a file. See the Quickstart and Usage.

AWS

infraharvest discover aws --profile prod --services=vpc,subnet,sg,s3 --regions=eu-west-2,us-east-1
infraharvest import aws --profile prod --services=vpc,subnet,sg,s3 --regions=eu-west-2,us-east-1 --selection=selection.yaml

For several accounts, add --accounts=111111111111,222222222222 or --organization: each account is reached through the role --assume-role names. See AWS.

Azure

export ARM_SUBSCRIPTION_ID=<subscription ID>
infraharvest discover azure --services=resource_graph --resource-group=web
infraharvest import azure --services=resource_graph --resource-group=web --selection=selection.yaml

Without --resource-group, the import covers the whole subscription. For several, use --subscriptions=<ID>,<ID> or --management-group=<group ID>. See Azure.

Google Cloud

infraharvest discover google --services='*' --projects=my-project --regions=global,europe-west2
infraharvest import google --services='*' --projects=my-project --regions=global,europe-west2 --selection=selection.yaml

--regions is global by default; import global with your regions. For several projects, use --projects=<ID>,<ID>, --organization=<organization ID> or --folders=<folder ID>,<folder ID>. See Google Cloud.

Apply the generated roots

infraharvest writes configuration and no state. Each root holds the generated configuration and imports.tf, with one import block per resource. You don't create or edit a state file first: the first terraform apply writes it.

cd infrastructure/aws/123456789012/eu-west-2
terraform init    # the backend in backend.tf, or local state without one
terraform plan    # imports every resource, changes nothing
terraform apply   # writes the resources into the state, changes nothing in the cloud
rm imports.tf     # the resources are in the state

• State: a backend (s3, azurerm or gcs) in the configuration file gives each root a backend.tf with a state key of its own, and infraharvest bootstrap --config <file> writes a root that creates the storage. Without one, each root keeps local state in terraform.tfstate.
• Secrets: infraharvest never writes a secret value. Each secret becomes a sensitive variable in variables.tf, without a default: set it before planning, from your secret store or with TF_VAR_<name>. The root's README lists them.
• Resources another state manages: import with --managed-state pointing at that state, and they are left out, so no resource ends up in two states.
• rejected.hcl: what Terraform couldn't import, with its errors. Terraform doesn't load it: fix a resource and move its blocks back, or leave it out.

See Applying the roots.

Day to day

The exit code is 0 when everything was imported, 1 when something wasn't (3 with --allow-partial), 2 when the import couldn't run, and 130 or 143 when it was interrupted or terminated (see Troubleshooting). Day to day has more.

Options that go together

• import needs --selection or --all, not both; discover refuses --all.
• --services is required, on the command line or in the configuration file.
• --incremental with --resume generates a root whose import didn't finish again, then adds to it.
• --regenerate or --incremental, not both; without either, an import into a root an earlier import finished is refused.
• --incremental finds what a root has from its import blocks or its checkpoint; without either, pass --managed-state.
• --managed-state=backend needs a backend in the configuration file.
• A --layout of your own starts with {out}/, contains {account} for several accounts, subscriptions or projects (on AWS, with any --accounts or --organization), and {env} with --env-tags.
• On AWS, --accounts or --organization, not both, and --assume-role with {account}.
• Azure's --subscriptions and --management-group, and Google Cloud's --projects, --organization and --folders, add up.

See Options that go together for the full list.

How it works

1. The provider's listers call the cloud's APIs and record each resource's type, ID and attributes. The catalogue turns them into the ID Terraform imports the resource by, and says which resources the default selection leaves out.
2. The selection decides which of them to import (see Choosing what to import).
3. infraharvest writes an import block for each one into a root, and runs terraform plan -generate-config-out (or tofu), so the provider itself reads every resource and writes its configuration.
4. It tidies the generated configuration: references between resources, shared tags and identifiers as locals, secrets as variables, and clusters of resources as module calls. Each change is kept only if a new plan shows no extra changes.
5. It runs the verification gate on every root and writes a report.

infraharvest reads the cloud and writes files; it never applies anything, and writes no state. terraform apply on a generated root records the imported resources in state without changing them.

Supported providers

• AWS
• Azure
• Google Cloud

Each is covered in depth: inventory APIs, the catalogue, curated modules, and end-to-end tests on an emulator on every pull request. Each is also tested against a real AWS account, Azure subscription and Google Cloud project with the read-only permission templates. A new provider is added by following the steps below.

Adding a provider

1. Create internal/providers/<name>/ with the provider and a lister for each of its services. The provider implements harvest.Provider (internal/harvest/provider.go), and Services maps each service's name to its lister, which usually embeds harvest.ListerOf. A lister's List(ctx, settings) lists the service's resources, records each one with harvest.Record or harvest.RecordBare: its ID, a name, the Terraform resource type, and attributes for import IDs and selection rules.
2. Create internal/cli/providers/provider_cmd_<name>.go, under the build constraint !minimal || <name>. It describes the provider's command with a definition (see command.go): the help, the provider's own flags, and the planner that turns them into the scopes and calls to import. It registers it with define from init().
3. Describe the provider in its Profile (internal/harvest/profile.go): its command's name, its registry Source when its Terraform provider isn't hashicorp/<name>, how it applies tags, which files of a root its resources go in, and what a root that environments share reads of its account and region. Implement the optional interfaces in internal/harvest/capabilities.go that it needs, for example ProviderWithScope to lay out roots by account and region, or ProviderWithTerraformEnv to pass credentials on to Terraform. For the query service, which lists through Terraform's own list resources, list harvest.QueryService among the services with a harvest.QueryLister, opt-in, set ListedTypes to the types its services list, which TestGoldenListedTypes (internal/cli) generates into the package's listed_types.go or, as AWS's services declare them, TestListedTypesAreDeclared checks, and implement ProviderWithQueryScope where the provider's scope is narrower than its provider block.
4. Add catalogue entries under internal/catalogue/data/<provider>/ for types that need them, such as an import ID template when the ID a lister records isn't the one Terraform imports by, each with a golden test (see docs/catalogue.md).
5. Add a page under docs/, list it in mkdocs.yml's navigation, and link it from this README. To have the command-line and services references cover it, add it to referenceProviders in internal/cli/clireference_test.go and rewrite them with the golden task of the remote workflow (see CONTRIBUTING.md).

Privacy

infraharvest collects no telemetry and sends nothing to its authors or anyone else. A run connects only to your clouds' APIs, the registries Terraform downloads providers and modules from, the state you point it at, and HashiCorp's release site when it has to download Terraform. Privacy lists every connection and every file it writes.

Contributing

Contributions are welcome. CONTRIBUTING.md describes the workflow: branches and pull requests against main, commit messages, the checks CI runs, and how to rewrite golden files and generated docs on GitHub without a local Go toolchain. docs/development.md is the reference for how the code is written: the layout, the conventions, the tests and the checks behind them. The documentation explains how infraharvest works from the user's side. Report vulnerabilities privately, as SECURITY.md says, and follow the code of conduct.

Licence

infraharvest is licensed under the GNU Affero General Public License, version 3 only (AGPL-3.0-only). The schema metadata it embeds, the defaults and argument constraints read out of the HashiCorp Terraform providers' code (internal/catalogue/schemameta/*.json.gz), is MPL-2.0, as those providers are (LICENSES/MPL-2.0.txt): NOTICE names the releases it was read from and where their source is. OPA's capabilities file it embeds (internal/engine/gate/conftest_capabilities.json) is Apache-2.0, as OPA is (LICENSES/Apache-2.0.txt), and NOTICE names its release too (see Licences).