Workbench v4.29.4

Schemathesis Workbench
Continuous fuzzing of open-source APIs.

ProfileBenchmark
1h budget per target, keeps testing after failures, no shrinking schemathesis run --max-time 3600 --continue-on-failure --no-shrink --suppress-health-check all <SCHEMA_URL>

Schema coverage measured with TraceCov, code coverage with coverage.py

Totalsacross all targets
12
APIs Tested
6,189
Issues Found
1,144,677
Total Requests
56.8%
Avg Code Coverage
9h 8m 3s
Total Runtime
Targets12 APIs

Open WebUI

FastAPI OpenAPI 3.1 154k
347,641 requests
1h 0m 0s duration
Self-hosted AI chat interface for local and remote models
2,094

Issues Found

527 of 535 API operations affected

See Report
108
Server Errors
2
Security
1,984
Contract Violations

Coverage

Timeline
Lines40.3%
18,767 / 46,559
Branches16.9%
2,708 / 16,046
Operations98.7%
528 / 535
Parameters98.3%
697 / 709
Keywords94.0%
4,177 / 4,442

API Responses

70,515
2xx (20.3%)
258,767
4xx (74.5%)
18,286
5xx (5.3%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Apache Airflow

FastAPI OpenAPI 3.1 47k
31,581 requests
1h 0m 0s duration
Platform to programmatically author, schedule and monitor workflows
261

Issues Found

88 of 128 API operations affected

See Report
27
Server Errors
231
Contract Violations
3
Other

Coverage

Timeline
Lines73.4%
8,437 / 11,498
Branches42.1%
1,196 / 2,844
Operations98.4%
126 / 128
Parameters97.3%
607 / 624
Keywords98.6%
2,413 / 2,447

API Responses

9,696
2xx (31.0%)
21,232
4xx (67.9%)
356
5xx (1.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Prefect

FastAPI OpenAPI 3.1 24k
29,012 requests
1h 0m 0s duration
Workflow orchestration framework for building resilient data pipelines in Python
530

Issues Found

173 of 187 API operations affected

See Report
57
Server Errors
471
Contract Violations
2
Other

Coverage

Timeline
Lines68.0%
14,844 / 21,820
Branches38.5%
2,104 / 5,468
Operations100.0%
187 / 187
Parameters88.4%
366 / 414
Keywords50.5%
8,516 / 16,852

API Responses

7,858
2xx (27.1%)
20,488
4xx (70.6%)
665
5xx (2.3%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Mealie

FastAPI OpenAPI 3.0 13k
67,499 requests
1h 0m 0s duration
Self-hosted recipe manager and meal planner with a RestAPI backend
804

Issues Found

238 of 247 API operations affected

See Report
92
Server Errors
712
Contract Violations

Coverage

Timeline
Lines73.5%
12,877 / 17,510
Branches42.7%
1,484 / 3,476
Operations96.4%
238 / 247
Parameters84.1%
626 / 744
Keywords89.8%
4,987 / 5,553

API Responses

27,953
2xx (42.0%)
31,354
4xx (47.2%)
7,182
5xx (10.8%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Polar

FastAPI OpenAPI 3.1 10k
102,162 requests
1h 0m 0s duration
Turn your software into a business - payment and monetization platform for open source
599

Issues Found

211 of 229 API operations affected

See Report
89
Server Errors
8
Security
501
Contract Violations
1
Other

Coverage

Timeline
Lines65.9%
19,900 / 30,218
Branches30.1%
1,891 / 6,282
Operations96.9%
222 / 229
Parameters53.7%
274 / 510
Keywords83.1%
4,543 / 5,465

API Responses

30,498
2xx (29.9%)
69,518
4xx (68.1%)
2,114
5xx (2.1%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

InvenTree

Django REST Framework OpenAPI 3.0 7.7k
33,949 requests
1h 0m 0s duration
Open source inventory management system with a Django REST Framework API
1,319

Issues Found

551 of 583 API operations affected

See Report
100
Server Errors
7
Security
1,212
Contract Violations

Coverage

Timeline
Lines59.2%
22,293 / 37,680
Branches23.3%
2,254 / 9,692
Operations98.6%
575 / 583
Parameters44.9%
986 / 2,197
Keywords35.8%
5,946 / 16,598

API Responses

8,118
2xx (23.9%)
24,285
4xx (71.5%)
1,542
5xx (4.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Weblate

Django REST Framework OpenAPI 3.1 6.1k
167,518 requests
1h 0m 0s duration
Web-based continuous localization system with a Django REST Framework API
339

Issues Found

125 of 160 API operations affected

See Report
17
Server Errors
303
Contract Violations
19
Other

Coverage

Timeline
Lines47.8%
23,211 / 48,512
Branches18.6%
2,635 / 14,168
Operations90.1%
145 / 161
Parameters56.3%
336 / 597
Keywords76.4%
4,513 / 5,905

API Responses

46,577
2xx (27.9%)
119,633
4xx (71.6%)
889
5xx (0.5%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

pygeoapi

Flask OpenAPI 3.0 626
180,929 requests
1h 0m 0s duration
OGC API server for geospatial data, built on Flask
192

Issues Found

63 of 66 API operations affected

See Report
14
Server Errors
178
Contract Violations

Coverage

Timeline
Lines61.1%
3,415 / 5,590
Branches46.2%
769 / 1,664
Operations98.5%
66 / 67
Parameters88.8%
190 / 214
Keywords93.3%
376 / 403

API Responses

81,580
2xx (45.1%)
95,640
4xx (52.9%)
3,566
5xx (2.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Openverse

Django REST Framework OpenAPI 3.0 380
175,293 requests
1h 0m 0s duration
Search engine API for openly licensed images and audio
29

Issues Found

12 of 17 API operations affected

See Report
2
Server Errors
27
Contract Violations

Coverage

Timeline
Lines78.0%
2,964 / 3,801
Branches49.8%
313 / 628
Operations100.0%
17 / 17
Parameters100.0%
70 / 70
Keywords100.0%
240 / 240

API Responses

17,472
2xx (10.0%)
157,697
4xx (90.0%)
21
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Covid 19 Japan Web API

Flask OpenAPI 2.0 129
2,383 requests
2m 1s duration
Web API to get COVID-19 information of each prefecture in Japan
1

Issue Found

1 of 4 API operations affected

See Report
1
Contract Violations

Coverage

Timeline
Lines95.2%
80 / 84
Branches87.5%
7 / 8
Operations100.0%
4 / 4
Parameters100.0%
3 / 3
Keywords100.0%
3 / 3

API Responses

1,564
2xx (65.6%)
819
4xx (34.4%)
0
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

REST-ler Blog API

Flask OpenAPI 2.0
6,062 requests
5m 0s duration
Demo API used in the original REST-ler paper
21

Issues Found

5 of 6 API operations affected

See Report
2
Server Errors
19
Contract Violations

Coverage

Timeline
Lines92.9%
171 / 184
Branches61.1%
11 / 18
Operations100.0%
6 / 6
Parameters100.0%
5 / 5
Keywords100.0%
6 / 6

API Responses

4,604
2xx (76.0%)
1,417
4xx (23.4%)
37
5xx (0.6%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Sample API

FastAPI OpenAPI 3.1.0
648 requests
1m 0s duration
A sample API designed to test the coverage effectiveness of Schemathesis tests
0

Issues Found

0 of 1 API operations affected

See Report

Coverage

Timeline
Lines100.0%
5 / 5
Operations100.0%
1 / 1

API Responses

643
2xx (99.2%)
5
4xx (0.8%)
0
5xx (0.0%)
Coverage Code | TraceCov
Stdout Web App | Schemathesis

Ready to test your API?

Start finding bugs in minutes with a single command

uvx schemathesis run <YOUR API SPEC URL>
Get Started Documentation

Want the same schema coverage for your own tests? Try TraceCov