Tags: mandiant/GoReSym
Tags
add support for Go runtime v1.26 (#85) * clarify Go runtime version vs GoReSym layout version * Add Go 1.26 support * Cleanup debugging prints * Remove remaining debug print in main.go * Fix redundant nil check in pe.go * Final touches: fix sudo in build script and rename layout convention to 1.22 * Run go fmt * Revert unnecessary moduledata scanning changes in executable parsers * refactor fixes * better variable name for refactor * Fix lots of version issues and bugs in magic prefix check, as well as concurrent lock bug * fmt * Refactor slice, textsec, and more to use offsets. Cleanup dead code in internal.go * fmt --------- Co-authored-by: Stephen Eckels <stevemk14ebr@gmail.com>
Add -strings flag to extract Go strings from binaries (#77) * Add -strings flag and ExtractMetadata field - Add Strings []string to ExtractMetadata struct - Add -strings command-line flag for string extraction - Update main_impl and main_impl_tmpfile signatures to accept printStrings parameter - Add placeholder string extraction logic with TODO marker - Update printForHuman to display extracted strings section - Verified flag appears in help and outputs correctly in both JSON and human format Part of #45 * Implement Go string extraction algorithm - Create objfile/strings.go with core extraction logic - Implement FLOSS-based string internment table detection - Add string candidate scanning (pointer + length pairs) - Implement findLongestMonotonicRun() for pattern detection - Add UTF-8 validation and printability filtering - Minimum string length: 4 characters, 80% printable - Add helper methods to elfFile: getSections(), is64Bit(), isLittleEndian() - Update main.go to call file.ExtractStrings() instead of placeholder - Tested with testproject/testproject: extracts 512 strings successfully - Extracts real Go strings: type names, runtime symbols, error messages Based on FLOSS floss/language/go/extract.py algorithm Part of #45 * Update README with -strings flag documentation - Add -strings flag to available flags list - Add Strings field to example JSON output - Document purpose: extract embedded Go strings from binary Part of #45 * Add tests validated against FLOSS output Per maintainer request, added comprehensive test suite: - strings_floss_test.go: Validates GoReSym against FLOSS reference output * 99.2% match rate (648/653 strings match FLOSS) * Uses FLOSS output from testproject.exe as ground truth * Reference saved in testdata/floss_reference.txt - strings_test.go: Additional unit tests for: * ELF and PE binary string extraction * Monotonic run detection algorithm * String filtering (printability, minimum length) - pe.go: Added helper methods (getSections, is64Bit, isLittleEndian) to enable string extraction from PE binaries All tests pass. * Address PR #77 review comments - Convert getSections() to iterateSections() using callback pattern to avoid memory pressure - Add Strings field to GoReSym.proto for external parsers - Implement iterateSections() for Mach-O format (previously missing) Changes requested by @stevemk14ebr in review: 1. Memory optimization: Replace array-based section loading with generator pattern 2. Proto definition: Add 'repeated string strings = 13' field 3. Mach-O support: Add missing iterateSections() implementation * Fix test failures and expand string extraction testing per PR #77 feedback - Fixed 4 test compilation errors by adding missing printStrings parameter to main_impl() calls - Added comprehensive TestStringExtraction function with 7 test cases covering Linux/macOS/Windows binaries - Implemented isPrintable() helper for ASCII validation (range 32-126) * Align string extraction with FLOSS algorithm per PR #77 review Rewrite string extraction to match FLOSS (extract.py) logic: - Sort candidates by address (not length) to fix monotonic run detection - Add image VA range and max section size filtering for candidates - Use candidate (pointer, length) pairs for direct extraction from blob - Replace 80% printable threshold with 100% fully printable check - Fix PE section addresses to include ImageBase for correct VA comparison Results: 648 strings extracted from PE test binary, 100% match with FLOSS. * Run string extraction on test build files * Extend main test to cover all new versions * Fix string extraction for old Windows binaries and add length sanity check 1. Add .text to data sections for old Go Windows binaries (1.7-1.10) that store strings in the code section instead of .rdata 2. Add maxReasonableStringLength (64KB) to filter out garbage candidates with huge lengths that cause incorrect blob boundary detection 3. Update TestIsDataSection to reflect .text being a valid data section Fixes string extraction failures on: - Old Windows PE binaries (Go 1.7-1.10): 0 -> 256 strings - Modern Windows: 574 strings - macOS: 284 strings All tests pass including TestWeirdBins on 15 real binaries. * Fix test failures for Go 1.5/1.6 and Go 1.22 macOS; fix build script typo 1. main_test.go: Skip Strings check for Go 1.5/1.6 Pre-SSA C-based linker does not produce length-sorted string blobs, so findLongestMonotonicRun never reaches the minimum threshold of 10. Pattern mirrors the existing interface-parsing guard. 2. objfile/strings.go: Handle prevNull == -1 in findStringBlobRange Apple's linker on newer Go/macOS (1.22+) packs sections without leading padding, so no null bytes exist before the first string candidate. bytes.LastIndex returns -1 in that case; treat it as offset 0 instead of bailing out with nil. 3. objfile/macho.go: Add missing is64Bit/isLittleEndian for machoFile elfFile and peFile already implement these interface methods; machoFile was the only rawFile implementation without them. Detect CPU type (CpuAmd64/Arm64 = 64-bit) and byte order from the macho.File struct. 4. build_test_files.sh: Fix $ver typo -> $GO_VER on mkdir line The directory was never created before Docker ran, causing Go 1.5 builds to produce no output. Now all 12 binaries are built for every version including 1.5 and 1.6. --------- Co-authored-by: Stephen Eckels <stevemk14ebr@gmail.com>
PreviousNext