Intro

As a Security Engineer at Cyera, one of my key responsibilities has been securing and enhancing our infrastructure. In this post, I'll share how I integrated Nuclei into our CI/CD pipeline to automate security testing for our API server.

The Challenge

When I joined Cyera, we needed a way to continuously test our API endpoints for security vulnerabilities without slowing down our development process. Manual security testing was time-consuming and couldn't keep pace with our rapid development cycles. We needed an automated solution that could:

1. Detect security vulnerabilities early in the development process
2. Integrate seamlessly with our existing CI/CD pipeline
3. Provide clear, actionable results for developers
4. Scale as our API surface area grew

Why Nuclei?

After evaluating several tools, I chose Nuclei for several reasons:

• Template-based approach: Nuclei uses YAML-based templates that are easy to customize
• Extensibility: We could write custom templates for our specific API endpoints
• Active community: Regular updates with new templates for emerging vulnerabilities
• Low false-positive rate: Compared to other tools we evaluated
• Performance: Fast scanning with minimal resource usage

Setting Up the Infrastructure

Step 1: Containerizing Nuclei

First, I needed to create a Docker container for Nuclei that could run in our CI/CD environment. Here's the Dockerfile I created:

FROM alpine:3.16
 
# Install required packages
RUN apk add --no-cache git go ca-certificates && \
    update-ca-certificates
 
# Install Nuclei
RUN go install -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei@latest
 
# Add Nuclei to PATH
ENV PATH="$PATH:/root/go/bin"
 
# Download Nuclei templates
RUN nuclei -update-templates
 
# Create directory for custom templates
RUN mkdir -p /custom-templates
 
# Set working directory
WORKDIR /scan
 
# Default command
ENTRYPOINT ["nuclei"]
CMD ["-h"]

This container includes Nuclei, its dependencies, and the official template repository. I also added a directory for our custom templates.

Step 2: Creating Custom Templates

Next, I wrote custom templates for our specific API endpoints. Here's an example of a template I created to test for authentication bypass:

id: auth-bypass-test
info:
  name: Authentication Bypass Test
  author: Conor Quinlan
  severity: high
  description: Tests for authentication bypass vulnerabilities in API endpoints
 
requests:
  - method: GET
    path:
      - "{{BaseURL}}/api/v1/protected-resource"
    headers:
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
      Accept: application/json
    matchers:
      - type: status
        status:
          - 200
      - type: word
        words:
          - "sensitive_data"
        part: body
    matchers-condition: and

I created dozens of custom templates covering various vulnerability types:

• Authentication and authorization flaws
• Injection vulnerabilities
• Information disclosure
• Rate limiting bypass
• Business logic flaws

Step 3: Integrating with CI/CD

I integrated Nuclei into our GitHub Actions workflow. Here's a simplified version of the workflow I created:

name: API Security Scan
 
on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]
 
jobs:
  nuclei-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      
      - name: Start API server for testing
        run: |
          docker-compose up -d api-server
          sleep 10  # Wait for server to start
      
      - name: Run Nuclei scan
        run: |
          docker run --network host \
            -v $(pwd)/custom-templates:/custom-templates \
            my-nuclei-image \
            -t /custom-templates,cves/ \
            -u http://localhost:8080 \
            -o nuclei-results.json \
            -j
      
      - name: Process scan results
        run: |
          python3 .github/scripts/process_nuclei_results.py
      
      - name: Upload scan results
        uses: actions/upload-artifact@v3
        with:
          name: nuclei-results
          path: nuclei-results.json
      
      - name: Fail if high severity issues found
        run: |
          if grep -q '"severity":"high"' nuclei-results.json; then
            echo "High severity issues found!"
            exit 1
          fi

This workflow:

1. Starts our API server in a test environment
2. Runs Nuclei with our custom templates
3. Processes the results
4. Fails the build if high-severity issues are found

Step 4: Result Processing and Reporting

To make the results more actionable, I wrote a Python script to process the Nuclei output and generate a more developer-friendly report:

#!/usr/bin/env python3
 
import json
import sys
from datetime import datetime
 
def process_results(input_file, output_file):
    with open(input_file, 'r') as f:
        results = [json.loads(line) for line in f]
    
    # Group by severity
    grouped = {
        "critical": [],
        "high": [],
        "medium": [],
        "low": [],
        "info": []
    }
    
    for result in results:
        severity = result.get("info", {}).get("severity", "info").lower()
        grouped[severity].append(result)
    
    # Generate summary
    summary = {
        "scan_date": datetime.now().isoformat(),
        "total_issues": len(results),
        "critical_count": len(grouped["critical"]),
        "high_count": len(grouped["high"]),
        "medium_count": len(grouped["medium"]),
        "low_count": len(grouped["low"]),
        "info_count": len(grouped["info"]),
        "results_by_severity": grouped
    }
    
    with open(output_file, 'w') as f:
        json.dump(summary, f, indent=2)
    
    # Print summary to console
    print(f"Total issues found: {summary['total_issues']}")
    print(f"Critical: {summary['critical_count']}")
    print(f"High: {summary['high_count']}")
    print(f"Medium: {summary['medium_count']}")
    print(f"Low: {summary['low_count']}")
    print(f"Info: {summary['info_count']}")
    
    # Exit with error if critical or high issues found
    if summary['critical_count'] > 0 or summary['high_count'] > 0:
        return 1
    return 0
 
if __name__ == "__main__":
    input_file = "nuclei-results.json"
    output_file = "nuclei-summary.json"
    sys.exit(process_results(input_file, output_file))

Results and Impact

After implementing this solution, we saw several significant benefits:

1. Early Detection: We caught 12 critical vulnerabilities before they reached production
2. Developer Education: The detailed reports helped developers understand security issues
3. Reduced Manual Testing: Automated 80% of our routine security tests
4. Faster Releases: Security testing no longer delayed our release cycles
5. Improved Security Posture: Overall reduction in vulnerabilities reaching production

Challenges and Lessons Learned

The implementation wasn't without challenges:

False Positives

Initially, we had issues with false positives. I addressed this by:

• Fine-tuning matcher conditions in templates
• Adding context-specific validation
• Implementing a feedback loop to improve templates

Performance Optimization

As our API grew, scan times increased. I optimized by:

• Parallelizing scans
• Using template filtering to run only relevant tests
• Implementing incremental scanning for PR changes

Developer Adoption

Getting developers to pay attention to security findings required:

• Creating clear, actionable reports
• Integrating results into existing developer tools
• Providing remediation guidance with each finding

Future Improvements

I'm continuing to enhance our security automation with:

1. Integration with JIRA: Automatically creating tickets for security issues
2. Custom Dashboard: Building a security dashboard for real-time visibility
3. Expanded Coverage: Adding more templates for new vulnerability types
4. Machine Learning: Exploring ML to reduce false positives and prioritize findings

Conclusion

Integrating Nuclei into our CI/CD pipeline has transformed how we approach security testing at Cyera. By automating security testing, we've shifted security left in our development process, catching vulnerabilities earlier and reducing the cost of remediation.

This project demonstrates how security automation can be both effective and developer-friendly, enabling teams to move fast without compromising security.

If you're interested in implementing a similar solution or have questions about my approach, feel free to reach out to me on LinkedIn or GitHub.

console.log("Hello World");