Conversation
This commit addresses multiple issues related to Table of Contents (TOC) generation, ensuring compatibility with Jekyll/Kramdown markdown processing when using Blackfriday.
The fixes include:
- **Recursive TOC Generation (CRITICAL)**: Prevented corruption by using unique placeholders to stop the HTML output from being reprocessed as markdown.
- **H1 Headings Included by Default**: Changed `toc_levels` default from "1..6" to "2..6" to match Jekyll's behavior of excluding H1 headings.
- **`{:.no_toc}` Marker Not Working**: Added logic to detect and ignore headings with `{:.no_toc}` markers, including checking the next sibling for the marker.
- **Ordered List Behavior Incorrect**: Ensured that `{:toc}` within `<ol>` tags is not processed for list replacement, matching Jekyll's behavior that only `<ul>` supports list replacement.
- **Code Block Protection**: Implemented mechanisms to protect TOC markers within code blocks (`<pre>`, `<code>`) so they are displayed literally rather than processed.
Additionally, comprehensive test cases have been added and existing ones expanded to cover these scenarios, including edge cases, special characters, and unusual heading hierarchies. Documentation has been updated to reflect the changes and provide clear verification steps.
The implementation strategy involves post-processing HTML generated by Blackfriday, rather than attempting to modify Blackfriday itself. This approach allows for Jekyll compatibility without forking the markdown processor.
Modified files:
- `renderers/markdown.go`
- `renderers/renderers.go`
- `renderers/markdown_toc_test.go`
Introduced files:
- `TOC-FIXES-SUMMARY.md`
- `TOC-TEST-RESULTS.md`
- `commands/README-TOC-TESTS.md`
- `commands/toc-edge-cases.md`
- `commands/toc-h1-test.md`
- `commands/toc-hierarchy-test.md`
- `commands/toc-special-chars.md`
- `commands/toc-test.md`
- `commands/toc-variations.md`
fix: Implement Jekyll-compatible TOC generation with Blackfriday
This commit refactors the TOC generation logic to align with Jekyll's Kramdown behavior when using the Blackfriday markdown processor. The core changes involve a shift from regex-based string manipulation to a DOM-based approach in `renderers/markdown.go` for more robust and accurate handling of TOC markers in various contexts.
Key changes include:
- **Removal of `TOC-FIXES-SUMMARY.md` and `TOC-TEST-RESULTS.md`**: These files summarized the fixes and test results, which are now implicitly covered by the code changes and commit history.
- **DOM-based TOC Processing**:
- TOC markers (`{:toc}`, `{::toc}`) and exclusion markers (`{:.no_toc}`) are now identified and manipulated within the parsed HTML document tree.
- This approach correctly handles marker context (e.g., within lists, code blocks, or standalone) and interactions between different markers.
- **Jekyll Compatibility**:
- `{:toc}` within an unordered list (`<ul>`) that contains only the marker is now correctly replaced with the generated TOC, mimicking Jekyll's behavior.
- `{:toc}` within ordered lists (`<ol>`) and `{::toc}` in any list context are no longer processed for list replacement, and are treated as standalone markers or ignored where appropriate, matching Jekyll's limitations.
- Headings with `{:.no_toc}` are correctly excluded from generated TOCs and removed from the output.
- Default `toc_levels` is now "2..6" to exclude H1 headings by default.
- **Code Block Protection**: Markers within `<pre>` and `<code>` tags are now correctly preserved and not processed as TOC directives.
- **Robust Marker Handling**:
- The `findTOCMarkersInDOM` function identifies all potential markers.
- `classifyMarkerContext` determines the precise context of each marker (e.g., `MarkerInUnorderedList`, `MarkerStandalone`, `MarkerInCodeBlock`).
- `replaceTOCMarkerInDOM` replaces markers based on their type, ensuring correct behavior for list replacements versus standalone replacements.
- **Code Structure Improvements**:
- `processTOC` now orchestrates the DOM parsing, marker finding, TOC generation, and DOM manipulation.
- Helper functions like `parseHTMLFragment`, `extractBodyContent`, `cloneNode`, `isOnlyContentInListItem`, and `isEmptyOrContainsNode` support the DOM processing.
- **Test Adjustments**:
- Unit tests in `renderers/markdown_toc_test.go` have been updated to reflect the new DOM-based processing logic and the specific behavior expected from Jekyll. For example, tests for ordered lists and block markers within lists now correctly expect no list replacement.
- **Related Issue**: This work addresses GitHub Issue #62 regarding TOC generation.
fix: Correct gojekyll TOC marker processing and HTML generation
This commit addresses several issues found in gojekyll's table of contents (TOC) generation logic, aligning its behavior with Jekyll 4.4.1.
The primary findings and implemented fixes are:
- `{::toc}` is not valid kramdown syntax and is not processed by Jekyll. gojekyll incorrectly processed this marker, leading to unexpected output. This has been corrected by removing `{::toc}` processing entirely. `tocPatternBlock` has been removed, and all TOC processing now relies solely on `tocPatternInline` (`{:toc}`).
- HTML corruption issues, where TOC divs appeared inside `<h2>` tags, have been fixed. This was a result of attempting to process `{:toc}` in inappropriate contexts. The `classifyMarkerContext` function has been updated to more accurately determine the context of TOC markers, ensuring that only valid `{:toc}` markers within unordered lists (`<ul>`) are processed. Standalone `{:toc}` markers and those in ordered lists (`<ol>`) are no longer processed, matching Jekyll's behavior.
- The handling of `{:.no_toc}` has been corrected. Previously, headings intended to be excluded from the TOC were still appearing. This was due to an issue in iterating through DOM siblings. The `extractHeadings` function now correctly skips over whitespace text nodes to find the relevant sibling element containing the `{:.no_toc}` marker.
- Documentation and test files have been updated to reflect the correct behavior of TOC markers as verified against Jekyll 4.4.1. This includes clarifying the invalidity of `{::toc}` and the specific conditions under which `{:toc}` is processed.
The changes ensure that gojekyll's TOC generation is accurate and consistent with Jekyll's standard behavior.
This commit significantly enhances the Table of Contents (TOC) generation logic within the markdown renderer, bringing it closer to the behavior of Jekyll.
Key changes include:
- **Jekyll Default Options**: The `TOCOptions` are now initialized with `UseJekyllHTML: true` by default, ensuring Jekyll-compatible HTML structure is preferred.
- **TOC Marker Processing**:
- The renderer now checks if TOC markers (`{:toc}`) are present within valid contexts (specifically, unordered lists or standalone on their own line) before attempting to process them. This aligns with Jekyll's behavior.
- If a `{:toc}` marker is found but not in a valid context, it is now removed from the HTML output, mimicking Jekyll's behavior of not rendering standalone markers.
- **`shouldProcessTOC` Function**: A new function is introduced to determine if the TOC should be processed based on the presence of markers in valid contexts.
- **`removeTOCMarkers` Function**: A new function is added to handle the removal of `{:toc}` markers that are not processed, preventing them from appearing as literal text in the output.
- **`MarkerStandalone` Type**: A new `MarkerContext` type, `MarkerStandalone`, is introduced to correctly identify TOC markers that appear on their own line.
- **`:.no_toc` Handling**: The logic for handling `:.no_toc` markers has been refined to only consider markers present within the heading element itself, not in subsequent sibling elements. This aligns with Jekyll's interpretation.
- **Test Updates**: Multiple test cases in `markdown_toc_test.go` have been updated to reflect the new Jekyll-compatible behavior, including changes to expected TOC output markers (from `<div class="toc">` to `<ul id="markdown-toc">`) and how standalone `{:toc}` markers are handled. Specifically, tests for standalone markers now expect them not to generate a TOC and instead be removed from the output.
- Move HTML attribute parsing and inner markdown rendering logic to new files: `markdown_attrs.go` and `markdown_toc.go`. - Consolidate common HTML utility functions (parsing, cloning, extracting body, rendering node to string, extracting text content) into `markdown_utils.go`. - This improves organization and modularity of the markdown rendering capabilities.
This commit refactors the TOC processing logic to better align with Jekyll's known behavior for `:toc` and `:.no_toc` markers.
- Adjustments to `renderers/markdown.go`:
- The logic for processing `:toc` markers is simplified. It now directly checks if `shouldProcessTOC` returns true, and if not, the marker is left as is (consistent with Jekyll not processing standalone `:toc`). Previously, it explicitly removed markers if they were not processed.
- The comment about Jekyll processing in unordered lists is clarified to reflect that standalone markers are not processed.
- Enhancements to `renderers/markdown_toc.go`:
- `processTOC` now defaults `UseJekyllHTML` to `true` if `TOCOptions` are not provided, ensuring Jekyll-compatible HTML structure by default.
- The `replaceTOCMarkerInDOM` function is updated to only replace markers within unordered lists (`<ul>`), aligning with Jekyll's behavior. Markers in code blocks, standalone contexts, ordered lists (`<ol>`), and other invalid contexts are now preserved literally.
- A new check `hasNoTocSibling` is introduced to correctly identify and exclude headings from the TOC when a `:.no_toc` marker is present in a sibling paragraph. This addresses cases where Kramdown places the IAL marker on the line after a heading, which Blackfriday renders as a separate `<p>` element.
- The test cases in `markdown_toc_test.go` are updated to reflect the refined behavior, particularly for standalone `{:toc}` and `:no_toc` markers.
- New tests added in `markdown_toc_test.go`:
- `TestLiteralTOCMarkers`: Verifies that `{:toc}` markers remain literal in headings, paragraphs, standalone paragraphs, and ordered lists, and that no TOC is generated in these cases.
- `TestNoTocSiblingParagraph`: Confirms that headings are excluded from the TOC when `:.no_toc` is used in a sibling paragraph or inline within the heading, aligning with improved `:.no_toc` handling.
- `TestMarkdownWithTOC`: Basic tests to ensure TOC generation works with Jekyll-style markdown.
- Updated existing tests to more accurately reflect Jekyll's behavioral nuances.
- Adjusted the spacing of struct fields in `markdown_toc_test.go` to align them vertically. - This change improves the readability of the test struct definition.
This was referenced Nov 22, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addresses multiple issues related to Table of Contents (TOC) generation.
The fixes include:
toc_levelsdefault from "1..6" to "2..6" to match Jekyll's behavior of excluding H1 headings.{:.no_toc}Marker Not Working: Added logic to detect and ignore headings with{:.no_toc}markers, including checking the next sibling for the marker.{:toc}within<ol>tags is not processed for list replacement, matching Jekyll's behavior that only<ul>supports list replacement.<pre>,<code>) so they are displayed literally rather than processed.Additionally, comprehensive test cases have been added and existing ones expanded to cover these scenarios, including edge cases, special characters, and unusual heading hierarchies. Documentation has been updated to reflect the changes and provide clear verification steps.
The implementation strategy involves post-processing HTML generated by Blackfriday, rather than attempting to modify Blackfriday itself. This approach allows for Jekyll compatibility without forking the markdown processor.