Intro

As a Security Engineer at Cyera, one of my key responsibilities has been securing and enhancing our infrastructure. In this post, I'll share how I integrated Nuclei into our CI/CD pipeline to automate security testing for our API server.

The Challenge

When I joined Cyera, we needed a way to continuously test our API endpoints for security vulnerabilities without slowing down our development process. Manual security testing was time-consuming and couldn't keep pace with our rapid development cycles. We needed an automated solution that could:

1. Detect security vulnerabilities early in the development process
2. Integrate seamlessly with our existing CI/CD pipeline
3. Provide clear, actionable results for developers
4. Scale as our API surface area grew

Why Nuclei?

After evaluating several tools, I chose Nuclei for several reasons:

• Template-based approach: Nuclei uses YAML-based templates that are easy to customize
• Extensibility: We could write custom templates for our specific API endpoints
• Active community: Regular updates with new templates for emerging vulnerabilities
• Low false-positive rate: Compared to other tools we evaluated
• Performance: Fast scanning with minimal resource usage

Setting Up the Infrastructure

Step 1: Containerizing Nuclei

First, I needed to create a Docker container for Nuclei that could run in our CI/CD environment. Here's the Dockerfile I created:

FROM alpine:3.16

# Install required packages
RUN apk add --no-cache git go ca-certificates && \
    update-ca-certificates

# Install Nuclei
RUN go install -v \
    github.com/projectdiscovery/nuclei/v2/cmd/nuclei@latest

# Add Nuclei to PATH
ENV PATH="$PATH:/root/go/bin"

# Download Nuclei templates
RUN nuclei -update-templates

# Create directory for custom templates
RUN mkdir -p /custom-templates

# Set working directory
WORKDIR /scan

# Default command
ENTRYPOINT ["nuclei"]
CMD ["-h"]

This container includes Nuclei, its dependencies, and the official template repository. I also added a directory for our custom templates.

Step 2: Creating Custom Templates

Next, I wrote custom templates for our specific API endpoints. Here's an example of a template I created to test for authentication bypass:

id: auth-bypass-test
info:
  name: Authentication Bypass Test
  author: Conor Quinlan
  severity: high
  description: Tests for authentication bypass vulnerabilities in API
  endpoints

requests:
  - method: GET
    path:
      - "{{BaseURL}}/api/v1/protected-resource"
    headers:
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
      AppleWebKit/537.36
      Accept: application/json
    matchers:
      - type: status
        status:
          - 200
      - type: word
        words:
          - "sensitive_data"
        part: body
    matchers-condition: and

I created dozens of custom templates covering various vulnerability types:

• Authentication and authorization flaws
• Injection vulnerabilities
• Information disclosure
• Rate limiting bypass
• Business logic flaws

Step 3: Integrating with CI/CD

I integrated Nuclei into our GitHub Actions workflow. Here's a simplified version of the workflow I created:

name: API Security Scan

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  nuclei-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Start API server for testing
        run: |
          docker-compose up -d api-server
          sleep 10  # Wait for server to start
      
      - name: Install Nuclei
        run: |
          # Use the official installation method with error handling
          curl -s \
            https://api.github.com/repos/projectdiscovery/nuclei/\
releases/latest \
            | grep "browser_download_url.*linux_amd64.zip" \
            | cut -d '"' -f 4 \
            | wget -qi -
          
          # Extract and install
          unzip -o -q nuclei_*_linux_amd64.zip
          sudo mv nuclei /usr/local/bin/
          sudo chmod +x /usr/local/bin/nuclei
          
          # Verify installation and update templates
          nuclei -version
          nuclei -update-templates
      
      - name: Start application for scanning
        working-directory: ./frontend
        run: |
          # Start the development server locally for security scanning
          bun run dev &
          echo "APPLICATION_PID=$!" >> $GITHUB_ENV
          
          # Wait for application to start
          echo "Waiting for application to start..."
          for i in {1..30}; do
            if curl -f -s http://localhost:3000 > /dev/null; then
              echo "Application is ready!"
              break
            fi
            echo "Waiting for application... (attempt $i/30)"
            sleep 2
          done
          
          # Final check
          curl -f http://localhost:3000 || \
            (echo "Application failed to start" && exit 1)
      
      - name: Run Nuclei scan
        run: |
          TARGET_URL="http://localhost:3000"
          echo "Running Nuclei scan against: $TARGET_URL"
          
          # Run comprehensive security scan
          nuclei -target "$TARGET_URL" \
            -tags cve,exposure,misconfig,vulnerability \
            -severity low,medium,high,critical \
            -jsonl -output nuclei-results.jsonl \
            -stats
          
          # Run web-specific scan
          nuclei -target "$TARGET_URL" \
            -tags tech,default-logins,web \
            -jsonl -output nuclei-web-results.jsonl \
            -stats
      
      - name: Process scan results
        run: |
          python3 .github/scripts/process_nuclei_results.py
      
      - name: Upload scan results
        uses: actions/upload-artifact@v3
        with:
          name: nuclei-results
          path: nuclei-results.json
      
      - name: Fail if high severity issues found
        run: |
          if grep -q '"severity":"high"' nuclei-results.json; then
            echo "High severity issues found!"
            exit 1
          fi

This workflow:

1. Starts our API server in a test environment
2. Runs Nuclei with our custom templates
3. Processes the results
4. Fails the build if high-severity issues are found

Step 4: Result Processing and Reporting

To make the results more actionable, I wrote a Python script to process the Nuclei output and generate a more developer-friendly report:

#!/usr/bin/env python3

import json
import sys
from datetime import datetime

def process_results(input_file, output_file):
    results = []
    with open(input_file, 'r') as f:
        for line in f:
            if line.strip():  # Skip empty lines
                results.append(json.loads(line))
    
    # Group by severity
    grouped = {
        "critical": [],
        "high": [],
        "medium": [],
        "low": [],
        "info": []
    }
    
    for result in results:
        severity = result.get("info", {}).get("severity", "info").lower()
        grouped[severity].append(result)
    
    # Generate summary
    summary = {
        "scan_date": datetime.now().isoformat(),
        "total_issues": len(results),
        "critical_count": len(grouped["critical"]),
        "high_count": len(grouped["high"]),
        "medium_count": len(grouped["medium"]),
        "low_count": len(grouped["low"]),
        "info_count": len(grouped["info"]),
        "results_by_severity": grouped
    }
    
    with open(output_file, 'w') as f:
        json.dump(summary, f, indent=2)
    
    # Print summary to console
    print(f"Total issues found: {summary['total_issues']}")
    print(f"Critical: {summary['critical_count']}")
    print(f"High: {summary['high_count']}")
    print(f"Medium: {summary['medium_count']}")
    print(f"Low: {summary['low_count']}")
    print(f"Info: {summary['info_count']}")
    
    # Exit with error if critical or high issues found
    if summary['critical_count'] > 0 or summary['high_count'] > 0:
        return 1
    return 0

if __name__ == "__main__":
    input_file = "nuclei-results.jsonl"
    output_file = "nuclei-summary.json"
    sys.exit(process_results(input_file, output_file))

Results and Impact

After implementing this solution, we saw several significant benefits:

1. Early Detection: We caught 12 critical vulnerabilities before they reached production
2. Developer Education: The detailed reports helped developers understand security issues
3. Reduced Manual Testing: Automated 80% of our routine security tests
4. Faster Releases: Security testing no longer delayed our release cycles
5. Improved Security Posture: Overall reduction in vulnerabilities reaching production

Challenges and Lessons Learned

The implementation wasn't without challenges:

False Positives

Initially, we had issues with false positives. I addressed this by:

• Fine-tuning matcher conditions in templates
• Adding context-specific validation
• Implementing a feedback loop to improve templates

Performance Optimization

As our API grew, scan times increased. I optimized by:

• Parallelizing scans
• Using template filtering to run only relevant tests
• Implementing incremental scanning for PR changes

Installation Issues

Initially, we encountered issues with the Nuclei installation in CI/CD environments. The original approach using hardcoded version URLs was brittle and prone to failures. I improved this by:

• Using the GitHub API to dynamically fetch the latest release URL
• Adding proper error handling and verification steps
• Ensuring executable permissions are set correctly
• Including template updates as part of the installation process
• Using non-interactive unzip flags to avoid prompts in CI/CD environments
• Using correct output format flags (-jsonl instead of -json)

The improved installation method is more robust and automatically adapts to new Nuclei releases without requiring manual updates to the CI/CD configuration.

Local vs Remote Testing

Initially, I attempted to scan Vercel preview deployments directly, but this introduced complexity around URL detection and deployment timing. I simplified the approach by:

• Starting the application locally with bun run dev
• Scanning against localhost:3000 consistently
• Eliminating dependency on external deployment URLs
• Reducing wait times and potential network issues

This local testing approach is more reliable, faster, and tests the same code that will be deployed, making it an ideal solution for CI/CD security scanning.

Developer Adoption

Getting developers to pay attention to security findings required:

• Creating clear, actionable reports
• Integrating results into existing developer tools
• Providing remediation guidance with each finding

Future Improvements

I'm continuing to enhance our security automation with:

1. Integration with JIRA: Automatically creating tickets for security issues
2. Custom Dashboard: Building a security dashboard for real-time visibility
3. Expanded Coverage: Adding more templates for new vulnerability types
4. Machine Learning: Exploring ML to reduce false positives and prioritize findings

Conclusion

Integrating Nuclei into our CI/CD pipeline has transformed how we approach security testing at Cyera. By automating security testing, we've shifted security left in our development process, catching vulnerabilities earlier and reducing the cost of remediation.

This project demonstrates how security automation can be both effective and developer-friendly, enabling teams to move fast without compromising security.

If you're interested in implementing a similar solution or have questions about my approach, feel free to reach out to me on LinkedIn or GitHub.

console.log("Hello World");