NFR document defines the performance expectations and testing requirements of the application. It acts as a baseline for planning, designing, executing and analyzing performance tests.
Purpose
Define performance goals, expectations and constraints for the application.
Involved
BA / Project Team, Architects, Infrastructure Team, NW Team, Stakeholders
Approval
Must be reviewed and approved by all stakeholders before test planning.
What does NFR include?
- CBT's (Critical Business Transactions)
- Peak Hours and Half Peak Hours (Low Load)
- Expected Average Response Times
- Number of Transactions per Hour
- Hardware Thresholds – CPU, Memory, Heap, Swapping
- Types of Tests: Availability, Serviceability, Scalability, Recoverability, Baseline, Benchmark, Failover, Abnormality, Spike
Note
NFR document must be prepared with the help of BA or project team and approved by all stakeholders.
Approach to Gather NFR
Scenario – 1
Application is already in production
- Get 1 year historical data using site analytics / Splunk / logs.
- Identify top 5 usage days.
- Find number of visitors, page views, top accessed pages (JSP/ASP), and access regions.
- Derive number of users and transactions.
- Identify CBT's from top pages.
- Conduct baseline test to identify expected response times.
Note:
If SLA/expected response times are not available, baseline test is mandatory.
Scenario – 2
Client doesn't know & app not in production
- Use competitor statistics tools (e.g., Alexa.com, SimilarWeb).
- Identify peak hours, half peak hours, users, page views, regions.
- Identify most accessed pages.
- Estimate expected response times for each page.
- Define NFR based on the findings.
Scenario – 3
No client data & no competitor available
- Understand the core business and convert it to online business.
- Estimate users, transactions, CBT's, and regions.
- Conduct baseline test to derive SLA's.
- Use BA questionnaire to estimate user base and peak load.
- Calculate users and transactions based on the answers.
NFR Requirement Template
| Category | Details to Be Captured | Web Application | API Application |
|---|---|---|---|
| Business | CBT's (Critical Business Transactions) | Page / User Journey | API Endpoints |
| Load Profile | Peak Hours, Half Peak Hours, Normal Load | Concurrent Users / Page Views | Transactions Per Second (TPS) |
| Volume | Transactions Per Hour (TPH) / Day | Page Views / Hits | API Calls / Transactions |
| Performance | Expected Average Response Time (SLA) | Page Response Time (sec) | API Response Time (sec) |
| Resource Thresholds | CPU Usage, Memory, Heap, Swapping, Disk I/O | Server / VM Level | Server / VM Level |
| Availability | Uptime %, Downtime Tolerance | Availability Requirement | Availability Requirement |
| Scalability | Expected Growth (Users / Transactions) | Concurrent Users | TPS / Throughput |
| Reliability | Recoverability, Failover, Data Consistency | Session / Data Handling | Error Handling / Retries |
| Types of Tests | Availability, Serviceability, Scalability, Recoverability, Baseline, Benchmark, Failover, Abnormality, Spike | Applicable Tests | Applicable Tests |
| Environment | Test Environment, Network, Dependencies | Web Servers, DB, NW | API Servers, DB, NW |
| Assumptions | Any assumptions considered | Browser, Network, Data | Client, Protocol, Data |
Example – Questionnaire to BA
| Question | Description | Example Answer |
|---|---|---|
| Q-1 | How many customers do you have in core business? | 1.3M customers |
| Q-2 | How many of them are going to register for online services? | 10% |
| Q-3 | How many of them are active users? | 10% |
| Q-4 | How many of them are daily users? | 30% |
| Q-5 | What are the peak hours for your business? | 4 hours |
Outcome
Based on the NFR, the project team can identify the right tool, protocol bundle, test approach, and estimate the effort (labor hours) required to complete performance testing.