Test Strategy for Startups: What to Test When You Can't Test Everything

At a startup, you don't have a 20-person QA team. You have 2 engineers and a deadline. You can't test everything.

The question isn't "should we test?" — it's "what do we test first?"

The Risk-Based Testing Pyramid

Forget the traditional testing pyramid (unit > integration > E2E). For startups, I use a risk-based approach:

Priority 1: Test things that lose money. Payment flows, subscription management, billing calculations. A bug here costs real dollars and real customers.

Priority 2: Test things that lose data. Database migrations, data exports, backup/restore. A bug here is catastrophic and often irreversible.

Priority 3: Test things that lose trust. Authentication, authorization, password reset, email delivery. A bug here makes users question your security.

Priority 4: Test everything else. UI interactions, edge cases, performance, accessibility. Important but not existential.

The Minimum Viable Test Suite

For a typical SaaS startup, here's what I'd set up in week 1:

tests/
├── smoke/           # Can the app start? (5 tests, 30 seconds)
│   ├── test_app_loads.py
│   └── test_api_health.py
├── critical_path/   # Can users sign up, pay, and use the product? (15 tests, 2 min)
│   ├── test_signup_flow.py
│   ├── test_payment_flow.py
│   └── test_core_feature.py
└── regression/      # Does existing functionality still work? (50+ tests, 5 min)
    ├── test_auth.py
    ├── test_api_endpoints.py
    └── test_data_integrity.py

Smoke tests run on every commit. Critical path runs on every PR. Regression runs nightly.

The "One Assertion" Rule

Every test should answer one question. Not three, not five. One.

# Bad: tests too many things at once
def test_user_registration():
    user = register("test@example.com", "password123")
    assert user.email == "test@example.com"
    assert user.is_active == True
    assert user.subscription_status == "trial"
    assert len(user.api_keys) == 1
    assert send_welcome_email.called == True
    assert analytics.track.called == True

# Good: one question per test
def test_user_registration_creates_active_user():
    user = register("test@example.com", "password123")
    assert user.is_active == True

def test_new_user_starts_on_trial():
    user = register("test@example.com", "password123")
    assert user.subscription_status == "trial"

When a multi-assertion test fails, you have to read the test to understand what broke. When a single-assertion test fails, the test name tells you.

When to Write Tests (The Practical Answer)

• Before fixing a bug: Write a test that reproduces the bug, then fix it. You'll never have that bug again.
• Before shipping a payment feature: Always. No exceptions.
• Before a refactor: Write tests for the current behavior, then refactor. The tests catch regressions.
• After a production incident: Write a test that would have caught it.

Don't write tests for trivial getters, UI layout, or code that changes daily. Spend your testing budget on stability, not coverage numbers.

The CI/CD Setup (15 Minutes)

name: CI
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.11' }
      - run: pip install -r requirements.txt
      - run: pytest tests/smoke/ -v
      - run: pytest tests/critical_path/ -v

That's it. 15 minutes to set up. Runs on every push. Catches the important stuff.

The Startup Testing Mindset

Testing at a startup isn't about 100% coverage. It's about sleeping at night knowing that:

1. Users can sign up
2. Payments work
3. Data isn't being lost
4. The app doesn't crash on load

Everything else is a luxury you earn with revenue.