Test Automation
Cypress
Github Actions
Mochawesome
Reports

Installation, Configuration, and Usage Guide for Mochawesome Reports

Massayoshi Pardo

13 min read5 days ago

Image 0: Mochawesome Test Report Dashboard

Mochawesome is a stylized test report for the popular JavaScript testing framework called Mocha. It provides a visually appealing and detailed presentation of test results, enabling developers and teams to gain clear insights into the quality of the tested code.

Mochawesome was created to address some limitations of the standard test reports provided by Mocha. With its user friendly interface and advanced features, Mochawesome offers a more enjoyable and informative test viewing experience.

One of the key advantages of Mochawesome is the generation of interactive HTML reports. The generated reports include an overview of test results, including information on tests that passed, failed, or were skipped. Additionally, the reports can display additional details such as error stacks, failure traces, and output messages, making debugging and issue understanding easier.

In addition to the visually appealing presentation, Mochawesome also supports advanced features such as result filtering, keyword searching, and individual test visualization. These features assist developers in quickly identifying problematic areas in their tests and taking corrective actions.

Mochawesome is highly configurable and can be easily integrated into existing test workflows. It is compatible with different testing environments, including browser testing, Node.js, and popular testing frameworks like Chai and Sinon.

In summary, Mochawesome is a powerful tool for generating test reports with Mocha. It offers an attractive visualization of test results, facilitating issue identification and improving the quality of the tested code.

Installation

The command npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator is used to install three packages related to Mochawesome: mochawesome, mochawesome-merge, and mochawesome-report-generator.

The mochawesome package is the core package of Mochawesome, responsible for generating the stylized test reports.
The mochawesome-merge package is used to merge multiple Mochawesome report files into a single file. This can be useful when you want to combine test results executed in different environments or multiple test runs.
The mochawesome-report-generator package is responsible for generating an interactive HTML report from the merged report files. It creates a user-friendly visualization of the test results, allowing you to navigate through the details of the executed tests.

So, type the following in the terminal:

npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator

When you run this command, npm will download and install these packages in your project.

By adding them as development dependencies (--save-dev), you can use these packages in the context of development and running tests in your project. If you remove the --save-dev option from the npm install mochawesome mochawesome-merge mochawesome-report-generator command, the packages will be installed as regular project dependencies rather than development dependencies.
This means that the packages will be listed in the package.json file of your project under the "dependencies" section instead of "devDependencies". Removing the --save-dev option can be useful if you plan to use the Mochawesome packages and its reporting tools not only during development and test execution but also in other scenarios, such as generating reports in a production environment or in CI/CD pipelines.

If everything goes well and you have executed the command correctly, you should see the following structure in the "devDependencies" section of your package.json file:

"devDependencies": {
    "mochawesome": "^8.1.1",
    "mochawesome-merge": "^4.4.1",
    "mochawesome-report-generator": "^6.2.0"
}

If you removed the --save-dev option when running the command, the resulting structure in the package.json file will be as follows:

"dependencies": {
    "mochawesome": "^8.1.1",
    "mochawesome-merge": "^4.4.1",
    "mochawesome-report-generator": "^6.2.0"
}

Local Configuration

Now we need to configure Mochawesome in the cypress.json.js file of Cypress. Please insert the following structure:

"reporter": "mochawesome",
"reporterOptions": {
    "reportDir": "mochawesome-report",
    "overwrite": false,
    "html": false,
    "json": true,
    "timestamp": "mmddyyyy_HHMMss"
}

Now, let’s explain each line of this configuration:

"reporter": "mochawesome": Specifies the reporter to be used for generating the test reports. In this case, we are setting the reporter as "mochawesome".
"reporterOptions": { ... }: Provides a series of configuration options for the Mochawesome reporter.
"reportDir": "mochawesome-report": Sets the directory where the Mochawesome reports will be generated. In this example, the reports will be saved in the "mochawesome-report" directory.
"overwrite": false: Determines whether existing reports should be overwritten or not. When set to false, the reports will not be overwritten, and a new version will be generated with a numeric suffix to avoid conflicts.
"html": false: Indicates whether an HTML report should be generated along with the JSON report. In this case, we are configuring it not to generate the HTML report by setting it to false.
"json": true: Determines whether the JSON report should be generated. When set to true, the JSON report will be generated.
"timestamp": "mmddyyyy_HHMMss": Defines a timestamp format that will be appended to the report file name. In this example, we are using the format "mmddyyyy_HHMMss", which will result in a timestamp in the "mmddyyyy_HHMMss" format being added to the report file name.

This configuration allows you to specify the output directory, control the generation of HTML and JSON reports, and customize the timestamp format in the report file names.

Folder Structure Creation

We need to create a new folder called "mochawesome-report" in the root of your project or wherever you prefer. However, make sure that the path of the "mochawesome-report" folder specified in the Mochawesome configuration in the cypress.json file matches the actual path where you created the folder. This will ensure that the reports are generated in the correct location. This rule also applies to the folder name. You can follow my example or create it according to your preference.

Running a test to generate the .json file

We need to run a test to generate the .json file, just to verify if the file will be successfully generated. It will also be useful when we run the scripts.

Make sure you are in the root directory of your project in the terminal or command prompt.
Execute the following command to start running the Cypress tests:

npx cypress run

Cypress will start executing the tests specified in the test files.

After the tests execution is completed, please check if the JSON file has been successfully generated in the "mochawesome-report" folder. For each completed test, a new .json file will be generated, and it will be displayed as shown in the image below.

If any issues occur during the execution of the command or if the JSON file is not generated as expected, please check the configuration of Cypress and Mochawesome to ensure that everything is properly set up in your project.

Scripts

Personally, I like to standardize the commands using 'scripts' within the 'package.json'. This also makes it easier for you to configure in GitLab for test execution in the pipeline. I will provide you with ready-made scripts to use in your project, but feel free to modify them according to your needs.

"scripts": {
    "cypress:merge-reports-json": "mochawesome-merge mochawesome-report/*.json -o mochawesome-report/report.json",
    "cypress:generate-report-html": "marge mochawesome-report/report.json",
    "cypress:merge-report-html": "mochawesome-merge > mochawesome-report/report.json",
    "cypress:generate-report-html-local": "marge mochawesome-report\\report.json",
    "cypress:merge-report-html-local": "mochawesome-merge > mochawesome-report\\report.json",
    "report:cleanup": "rm -f mochawesome-report/mochawesome_*.json"
}

The difference between the pipeline and local scripts lies in the slashes used (/ and \), which are related to how file paths are specified in different operating systems.

In the Continuous Integration (CI) environment, which is typically run on Unix-based systems like Linux, the forward slash (/) is used to separate directories in file paths. For example, "mochawesome-report/*.json" specifies that all files with the .json extension in the mochawesome-report directory should be included.

On the other hand, in the local environment, which is usually run on Windows systems, the backslash (\) is used to separate directories in file paths. For example, "mochawesome-report\report.json" specifies the file report.json in the mochawesome-report directory.

This difference arises from the conventions of each operating system regarding file path representation.

Now, let’s explain each script in the package.json file.

cypress:merge-reports-json

The cypress:merge-reports-json script is responsible for merging the JSON files of the reports generated by Mochawesome.

To execute the script, use the following command:

npm run cypress:merge-reports-json

When you run the command for the first time, the report.json file will be created. Then, the file will be updated with each subsequent execution.

cypress:generate-report-html

The cypress:generate-report-html script generates the final HTML report based on the merged JSON file.

To execute the script, use the following command:

npm run cypress:generate-report-html

When you run the command for the first time, the report.html file and the assets folder with all the files will be created. Then, the report.html file will be updated with each subsequent execution.

Image 4: Report.html File Generation and Assets Folder

cypress:merge-report-html

The cypress:merge-report-html script merges the HTML reports generated by Mochawesome.

Get Massayoshi Pardo’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

To execute the script, use the following command:

npm run cypress:merge-report-html

When you run the command, the report.html file is updated with the information from all the tests.

report:cleanup

The report:cleanup script is used to delete files with a specific pattern "mochawesome_*.json". It will delete all files that start with "mochawesome_" and have the extension ".json".

To execute the script, use the following command:

npm run report:cleanup

Scripts Pipeline e Local

This table presents a division of the scripts into two categories: Pipeline and Local.

The scripts marked with "Local" should be executed for local tests. They also merge JSON report files, generate HTML reports, and merge HTML report files, but for the local environment.

Make sure to run the scripts corresponding to the appropriate category, depending on the context you are working in: either in the CI pipeline or for local tests.

Folder Structure

The main files to be aware of are:

report.html: This is the test report generated in HTML format. It provides a detailed view of the test results obtained using Mochawesome. The report.html file displays information about test suites, individual test cases, test outcomes (pass, fail, pending), error messages, and other relevant details. Additionally, the report may include charts, statistics, and interactive features to facilitate result analysis.

report.json: This file is a JSON file that contains raw test result data. It stores the same information present in report.html but in a structured data format. The report.json file can be useful if you want to process and analyze the test result data in a machine-readable format or generate custom reports using specific tools or scripts.

mochawesome_06272023_204500.json: The .json file is an example of an individual result file generated by Mochawesome for each test run. The specific value "06272023_204500" in the filename represents the date and time of the test execution, and it will be different for each test run.

When you execute your tests using Mochawesome, the framework generates a separate JSON result file for each test run. These files follow a specific naming format, typically including the date and time of the test execution.

The content of the mochawesome_06272023_204500.json file may vary depending on the structure and details of your tests, but it generally contains the following information:

General information: The file may include general information about the test execution, such as the date and time when the tests were run, the total duration of the test run, and other relevant metadata.
Test suites: The file may contain information about the test suites that were executed. A test suite is a collection of related test cases that test a specific functionality or component of the system.
Test cases: The file may provide details about each individual test case that was executed. This includes information such as the test case name, test status (pass, fail, pending), success or error messages, execution time, and other relevant information for each test case.
Statistics: The file may include aggregated statistics about the test execution, such as the total number of tests executed, the number of tests that passed, failed, or are pending, and other performance-related metrics.

These individual result files are automatically generated by Mochawesome during test execution. They can be used for detailed result analysis, generating custom reports, or tracking the test execution history over time.

Execution Sequence in pipeline

It’s important to run the scripts in the following order for pipeline:

cypress:merge-reports-json
cypress:generate-report-html
cypress:merge-report-html

Execution Sequence in Local

cypress:merge-reports-json
cypress:generate-report-html-local
cypress:merge-report-html-local

By running the scripts correctly, you can navigate to the mochawesome-report folder and you will see the following image.

Click on the report.html file and witness the magic happen.

Pipeline Configuration

After setting up everything locally, we now need to configure Mochawesome in the GitHub Actions workflow file located in .github/workflows/. The logic will be the same, we just need to configure the workflow to identify and generate the files according to the configuration. Open your workflow YAML file and configure the steps responsible for running the tests, generating the reports, and uploading the artifacts.

# ------------------------------------------------------------------------------
# Consolidated Reports
# ------------------------------------------------------------------------------

reports:
  name: Consolidate QA Reports
  runs-on: ubuntu-latest

  needs: test-api-qa
  if: always()

  steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: 20
        cache: npm
        cache-dependency-path: api/package-lock.json

    - name: Install dependencies
      working-directory: api
      run: npm ci

    # -------- DOWNLOAD REPORTS --------

    - name: Download Mochawesome raw reports
      uses: actions/download-artifact@v4
      with:
        pattern: cypress-raw-*
        path: api/mochawesome-report
        merge-multiple: true

    - name: Download Cypress screenshots
      uses: actions/download-artifact@v4
      with:
        pattern: cypress-screenshots-*
        path: api/cypress/reports/screenshots
        merge-multiple: true

    - name: List downloaded report files
      working-directory: api
      run: ls -al mochawesome-report/

    # -------- GENERATE REPORTS --------

    - name: Merge Mochawesome JSON reports
      working-directory: api
      run: npm run cypress:merge-reports-json

    - name: Generate Mochawesome HTML
      working-directory: api
      run: npm run cypress:generate-report-html

    - name: Merge Mochawesome HTML
      working-directory: api
      run: npm run cypress:merge-report-html

    - name: List generated report files
      working-directory: api
      run: ls -al mochawesome-report/

    # -------- UPLOAD FINAL ARTIFACTS --------

    - name: Upload consolidated Mochawesome HTML report
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: mochawesome-report
        path: api/mochawesome-report/
        if-no-files-found: warn
        retention-days: 7

    - name: Upload consolidated JUnit XML
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: junit-report
        path: api/mochawesome-report/junit-report-*.xml
        if-no-files-found: warn
        retention-days: 7

    - name: Upload consolidated Cypress screenshots
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: cypress-screenshots
        path: api/cypress/reports/screenshots/
        if-no-files-found: warn
        retention-days: 7

Explaining the added structure:

needs: test-api-qa: Ensures that the report consolidation job runs after the API test job has completed.
if: always(): Allows the report consolidation job and artifact upload steps to run even if a previous job or step fails. This is useful when you need to access test results even when the workflow does not complete successfully.
actions/download-artifact@v4: Downloads the raw Mochawesome reports and Cypress screenshots generated by the test jobs.
npm run cypress:merge-reports-json: Merges the individual Mochawesome JSON report files into a consolidated JSON report.
npm run cypress:generate-report-html: Generates the HTML report from the consolidated JSON report.
npm run cypress:merge-report-html: Executes the additional HTML consolidation script configured in package.json.
actions/upload-artifact@v4: Stores the generated reports and screenshots as GitHub Actions artifacts, allowing you to download and inspect them later.
retention-days: 7: Specifies that the uploaded artifacts will be retained for seven days.

Once the workflow finishes, open the Actions tab in your GitHub repository, select the corresponding workflow run, and locate the Artifacts section to download the generated reports.

Image 10: json file generated in the pipeline

At the end of each test, a .json file will be generated.

Image 11: Executing the scripts in the pipeline

At the end of the workflow execution, the steps responsible for generating and consolidating the reports will be executed.

To view the generated files, access the Artifacts section at the bottom of the workflow run summary page in GitHub Actions. Download the desired artifact to access the generated reports.

By clicking on the artifact name in the Artifacts section, you can download the generated files as a compressed ZIP file to your machine.

Extract the downloaded ZIP file into your project directory.

Open the report.html file in your browser to view the generated report, as shown in the image below.

This brings us to the end of our guide! We hope it has helped you install and configure Mochawesome reports in your project.

If you have any questions, find any errors in the documentation, or have suggestions for improvements, please feel free to contact me. I’m always open to feedback and happy to exchange ideas about test automation and software quality engineering.