Sample test report for trading software
This is a sample of the format, and it is not a client project. Instead of inventing numbers for a fictional client, it reports a real run of Fincode's published teaching reference, an order gateway, against its own mock exchange on one computer. No order reached any exchange, the mock is not a live market, and nothing here is a trading result.
Sample format, real run, no client. The software under test is the order gateway reference, which you can download and run with the same commands. Every count, test name and output line below comes from runs on November 5, 2026: of the published download, and in section 7 of an earlier version of the same code. Times will differ on your machine. A report for your own project follows this format and is what you receive in the third stage of how we work.
| Report | Test report |
|---|---|
| Software | Order gateway, a teaching reference in Python, standard library only. Not production software and not a product |
| Version tested | The download fincode-order-gateway-reference.zip as of November 5, 2026, SHA-256 bb110bc1098e2e9209e344b7eb7325ff528a99617e54900b5ef4a23871785e59, unzipped into an empty folder |
| Requirements | The behavior the reference's README states: when a retry is safe, the order states and the design decisions |
| Run on | November 5, 2026 |
| Prepared by | Fincode |
| Prepared for | No client. Published as a sample |
1. Summary
All 91 automated tests passed on Windows and on Linux, and all 10 demo scenarios behaved as the README describes. The demo printed the same text on both systems and on repeated runs. Of the 14 failure cases this report format covers, 7 passed, 3 were exercised only in part and 4 were not run, because the reference leaves those parts out on purpose. Run against an earlier version of the same code, 13 of today's tests fail, and the defects behind them are listed in section 7.
| Automated tests | 91 run, 91 passed, on each of the two systems |
|---|---|
| Demo scenarios | 10 run, 10 behaved as expected |
| Failure cases T01 to T14 | 7 passed, 3 passed in part, 4 not run |
| Rule checks W01 and P01 | Not run. The reference has no trading rules |
| Open defects | 0 |
| Orders sent to a live exchange | 0 |
| Profit, return or win-rate figures | None. A test report does not contain them |
2. Test environment
| System 1 | Python 3.11.3 on Windows 11 (version 10.0.26300) |
|---|---|
| System 2 | Python 3.12.13 on Debian 13, in a Docker container from the python:3.12-slim image, started with --network none, so loopback was the only network interface |
| Exchange side | The reference's own mock exchange, gateway/mock_exchange.py, listening on 127.0.0.1. It has no order book, no prices and no liquidity: an order rests until a test calls apply_fill() |
| Fault injection | Scripted in the mock: 5xx answers, 429 with or without Retry-After, a lost or garbled answer after the order was taken, a timeout, a read that cannot see the order yet, and a stale order status |
| Crashes | A child process is killed with os._exit() at a named point, so the next process sees only what reached the disk |
| Waits | Sleeps in tests go through an injected clock, and backoff has no jitter, so runs repeat exactly |
| API keys | None. The reference has no keys and no request signing, and its client refuses any host but this machine |
| Risk limits | The values in EXAMPLE_LIMITS, which exist so the tests have something to exercise. They are not recommended settings |
| Commands | python -m unittest -v, python demo.py twice, and python -X dev -W error -m unittest, which also passed on both systems |
> python -m unittest -v
test_concurrent_duplicates_send_one_order (tests.test_gateway.DuplicateRequest.test_concurrent_duplicates_send_one_order) ... ok
test_duplicate_request_sends_one_order (tests.test_gateway.DuplicateRequest.test_duplicate_request_sends_one_order) ... ok
[89 more lines, all ending in ok]
----------------------------------------------------------------------
Ran 91 tests in 12.955s
OKThat is the Windows run. On Linux the last lines read Ran 91 tests in 8.269s and OK, with the tests in the same order.
3. Results at a glance
Each case names the demo scenario and the test class that exercise it. A case the reference does not exercise is marked as not run, with the reason.
| ID | Case | Exercised by | Result |
|---|---|---|---|
| T01 | Duplicate delivery | Demo duplicate_request; 4 tests in DuplicateRequest | Pass |
| T02 | Exchange 5xx | Demos server_error_order_absent and server_error_order_taken; 7 tests in ExchangeServerError | Pass |
| T03 | Late message | Nothing. The reference has no message age check | Not run |
| T04 | Invalid payload | 7 tests in TypedRequest. There is no network intake, so signature and size checks do not exist to test | Pass, in part |
| T05 | Oversized quantity | Demo oversized_quantity; 9 tests in RiskLimitsOutsideTheRules | Pass |
| T06 | Dropped connection | Demo lost_response; 9 tests in LostResponse and 2 on an unreachable exchange. No streaming connection exists to drop | Pass, in part |
| T07 | Restart mid-order | Demos crash_after_send and crash_before_send; 17 tests in RestartRecovery | Pass |
| T08 | Rate-limit response | Demo rate_limit; 7 tests in RateLimit | Pass |
| T09 | Partial fill | Demo partial_fill; 9 tests in PartialFill | Pass |
| T10 | Clock offset | Nothing. No request is signed, so there is no clock check | Not run |
| T11 | Emergency stop | Demo kill_switch; 10 tests in KillSwitchBehavior | Pass |
| T12 | Secrets in logs | Nothing. The reference holds no key or secret that could leak | Not run |
| T13 | Reconciliation mismatch | Orphan and mismatch tests in RestartRecovery. Balances are not reconciled | Pass, in part |
| T14 | Symbol rules | Nothing. Tick size, step size and minimum order value are left out on purpose | Not run |
4. Failure cases in detail
Failures are scripted on purpose, because a test environment rarely fails at the moment a test needs it to. The log excerpts are the demo's own output, unedited. Each numbered line is one event from the JSON lines log, printed without its timestamp and a few other fields, so that the text repeats exactly. python demo.py --raw prints the full lines.
T01 Duplicate delivery
| Setup | The same request, with the same request ID, was submitted three times. A test also submits it from eight threads at once. |
|---|---|
| Expected | One order. A repeat gets the stored result and sends nothing. |
| Observed | One POST and one order at the mock exchange. Both repeats were logged as duplicate_request. The concurrent test also ended with one order. |
| Tests | test_concurrent_duplicates_send_one_order, test_duplicate_request_sends_one_order, test_same_amount_written_differently_is_the_same_request, test_same_request_id_with_different_order_is_refused |
| Result | Pass. |
== duplicate_request: The same request arrives three times
1 request_received request_id=demo-1 side=BUY quantity=1 limit_price=20
2 order_journaled state=SUBMITTING
3 send_attempt attempt=1
4 send_result outcome=OK http_status=200 attempt=1
5 state_changed from_state=SUBMITTING to_state=OPEN
6 request_received request_id=demo-1 side=BUY quantity=1 limit_price=20
7 duplicate_request state=OPEN
8 request_received request_id=demo-1 side=BUY quantity=1 limit_price=20
9 duplicate_request state=OPEN
result: state=OPEN filled=0 reason=None POST calls=1 orders at exchange=1T02 Exchange 5xx
| Setup | Variant (a): the mock answered 503 and did not take the order. Variant (b): the mock took the order and answered 500. Tests add 5xx followed by status reads that fail, a one-attempt policy, repeated 5xx and a 503 that carries Retry-After. |
|---|---|
| Expected | The order becomes UNKNOWN, and nothing is sent again before a status read. If two reads find no order, it is sent once more with the same client order ID. If a read finds it, it is recorded and not sent again. If no read gives a usable answer, it stays UNKNOWN and new orders are refused. |
| Observed | (a) Two reads answered 404, then one more send: 2 POSTs, 1 order. (b) The first read found the order: 1 POST, 1 order. |
| Tests | 7 in ExchangeServerError, including test_5xx_then_status_unavailable_is_unknown_not_failed and test_unknown_order_blocks_new_orders_until_reconciled |
| Result | Pass. |
== server_error_order_absent: The exchange answers 503 and did not take the order
1 request_received request_id=demo-2 side=BUY quantity=1 limit_price=20
2 order_journaled state=SUBMITTING
3 send_attempt attempt=1
4 send_result outcome=SERVER_ERROR http_status=503 attempt=1
5 state_changed from_state=SUBMITTING to_state=UNKNOWN reason=SERVER_ERROR
6 status_read outcome=NOT_FOUND http_status=404 read=1
7 retry_wait seconds=0.2 because=NOT_FOUND
8 status_read outcome=NOT_FOUND http_status=404 read=2
9 retry_wait seconds=0.2 because=SERVER_ERROR
10 state_changed from_state=UNKNOWN to_state=SUBMITTING
11 send_attempt attempt=2
12 send_result outcome=OK http_status=200 attempt=2
13 state_changed from_state=SUBMITTING to_state=OPEN
result: state=OPEN filled=0 reason=None POST calls=2 orders at exchange=1
== server_error_order_taken: The exchange answers 500 but did take the order
1 request_received request_id=demo-3 side=BUY quantity=1 limit_price=20
2 order_journaled state=SUBMITTING
3 send_attempt attempt=1
4 send_result outcome=SERVER_ERROR http_status=500 attempt=1
5 state_changed from_state=SUBMITTING to_state=UNKNOWN reason=SERVER_ERROR
6 status_read outcome=OK http_status=200 read=1
7 state_changed from_state=UNKNOWN to_state=OPEN
result: state=OPEN filled=0 reason=None POST calls=1 orders at exchange=1Sending again after two reads find nothing is safe only on a platform that enforces unique client order IDs, and only for as long as it enforces them. The mock answers a second POST with a known client order ID with 409. A real project checks the platform's own documentation, and where uniqueness is not guaranteed the reference can be set to one attempt, so an unclear send ends in a status read.
T03 Late message
Not run. The gateway takes typed requests from code in the same process, so there is no incoming message with a creation time to check. A project that receives alerts or webhooks needs this case: a message older than the age the customer sets is rejected and logged, and no order is created.
T04 Invalid payload
| Setup | Requests built with a float amount; amounts of NaN, infinity, minus one, zero, 1E+30 and 0.0000000000001; an unknown or missing field; a bad side or request ID; and data that is not a mapping at all. |
|---|---|
| Expected | Each is refused with InvalidOrderRequest before anything is journaled or sent. A valid mapping becomes a typed request. |
| Observed | All refused as expected. The client order ID derived from a request ID was the same on every call. |
| Tests | test_bad_amounts_are_refused, test_bad_side_and_request_id_are_refused, test_client_order_id_is_derived_and_stable, test_data_that_is_not_a_mapping_is_refused, test_float_amount_is_refused, test_unknown_or_missing_field_is_refused, test_valid_mapping_becomes_a_typed_request |
| Not run | Malformed JSON, a body changed after signing and a body over a size limit. The reference has no network intake, so these checks do not exist in it. |
| Result | Pass, in part. |
T05 Oversized quantity
| Setup | A quantity above the example limit. Tests add the notional limit, the position limit on both sides, open orders counted as filled, concurrent orders racing the position limit, a symbol not on the allowed list and a gateway opened without limits. |
|---|---|
| Expected | Blocked before the exchange, with the reason logged. A gateway cannot be opened without limit values. |
| Observed | Blocked with quantity_above_limit. No POST reached the mock exchange. |
| Tests | 9 in RiskLimitsOutsideTheRules, including test_concurrent_orders_cannot_overrun_the_position_limit and test_limits_have_no_default_values |
| Not run | A daily loss limit and a limit on orders per minute. The reference has neither. |
| Result | Pass. |
== oversized_quantity: The quantity is above the risk limit
1 request_received request_id=demo-7 side=BUY quantity=11 limit_price=20
2 order_blocked reason=quantity_above_limit
result: state=BLOCKED filled=0 reason=quantity_above_limit POST calls=0 orders at exchange=0T06 Dropped connection
| Setup | The mock took the order, then closed the connection without answering. Tests add timeouts with and without the order taken, a garbled 200, an order that becomes visible to reads late, an exchange that cannot be reached at all, and start-up while it is down. |
|---|---|
| Expected | A lost answer makes the order UNKNOWN, and a status read settles it. A connection that never opened means nothing was sent. |
| Observed | The first read found the order: 1 POST, 1 order. In the tests, an unreachable exchange left no order and the gateway stayed closed. |
| Tests | 9 in LostResponse, plus test_exchange_down_means_not_sent_and_no_order and test_startup_with_exchange_down_stays_closed |
| Not run | A dropped or silent streaming connection, reconnect backoff and resubscribing. The reference reads fills by polling and has no stream. |
| Result | Pass, in part. |
== lost_response: The exchange takes the order and the response is lost
1 request_received request_id=demo-5 side=BUY quantity=1 limit_price=20
2 order_journaled state=SUBMITTING
3 send_attempt attempt=1
4 send_result outcome=CONNECTION_LOST attempt=1
5 state_changed from_state=SUBMITTING to_state=UNKNOWN reason=CONNECTION_LOST
6 status_read outcome=OK http_status=200 read=1
7 state_changed from_state=UNKNOWN to_state=OPEN
result: state=OPEN filled=0 reason=None POST calls=1 orders at exchange=1T07 Restart mid-order
| Setup | A child process was killed with os._exit() at two points: (a) after the POST, before the answer was recorded, and (b) after the journal write, before the POST. A new process then reconciled, and the same request was submitted again. Tests add fills during the downtime, orphan orders, an exchange that is down at start-up, and an order that arrives after it was declared not placed. |
|---|---|
| Expected | New orders are refused until reconciliation comes back clean. (a) The order is found and adopted. (b) Two reads find nothing, the order becomes NOT_PLACED, and it is never sent later. The repeated request sends nothing. |
| Observed | (a) A new request during the gap was blocked with not_reconciled. Reconciliation found the order OPEN. 1 POST, 1 order. (b) NOT_PLACED after two 404 reads. 0 POSTs, 0 orders. |
| Tests | 17 in RestartRecovery, including test_restart_refuses_new_orders_until_reconciled and test_order_arriving_after_not_placed_is_found_and_canceled |
| Result | Pass. |
== crash_after_send: The process dies after the POST, before recording the answer
1 reconcile_started orders_to_check=0
2 working_orders_listed outcome=OK count=0
3 reconcile_finished clean=True resolved=0 unresolved=0 orphans=0 late_orders=0
4 request_received request_id=demo-10 side=BUY quantity=1 limit_price=20
5 order_journaled state=SUBMITTING
6 send_attempt attempt=1
7 send_result outcome=OK http_status=200 attempt=1
8 request_received request_id=demo-11 side=BUY quantity=1 limit_price=20
9 order_blocked reason=not_reconciled
10 reconcile_started orders_to_check=1
11 status_read outcome=OK http_status=200 read=1
12 state_changed from_state=SUBMITTING to_state=OPEN
13 working_orders_listed outcome=OK count=1
14 reconcile_finished clean=True resolved=1 unresolved=0 orphans=0 late_orders=0
15 request_received request_id=demo-10 side=BUY quantity=1 limit_price=20
16 duplicate_request state=OPEN
result: state=OPEN filled=0 reason=None POST calls=1 orders at exchange=1
== crash_before_send: The process dies after the journal write, before the POST
1 reconcile_started orders_to_check=0
2 working_orders_listed outcome=OK count=0
3 reconcile_finished clean=True resolved=0 unresolved=0 orphans=0 late_orders=0
4 request_received request_id=demo-12 side=BUY quantity=1 limit_price=20
5 order_journaled state=SUBMITTING
6 reconcile_started orders_to_check=1
7 status_read outcome=NOT_FOUND http_status=404 read=1
8 retry_wait seconds=0.2 because=NOT_FOUND
9 status_read outcome=NOT_FOUND http_status=404 read=2
10 state_changed from_state=SUBMITTING to_state=NOT_PLACED reason=absent_at_exchange
11 working_orders_listed outcome=OK count=0
12 reconcile_finished clean=True resolved=1 unresolved=0 orphans=0 late_orders=0
13 request_received request_id=demo-12 side=BUY quantity=1 limit_price=20
14 duplicate_request state=NOT_PLACED
result: state=NOT_PLACED filled=0 reason=absent_at_exchange POST calls=0 orders at exchange=05. Other checks
| Check | How it was run | Observed | Result |
|---|---|---|---|
| T08 Rate-limit response | Demo rate_limit: one 429 with Retry-After: 1. 7 tests in RateLimit, including an HTTP date, no header at all, and a wait past the deadline | Logged a wait of 1.0 s, then sent again: 2 POSTs, 1 order. The tests check that the same client order ID goes out again, and that nothing more is sent past the deadline | Pass |
| T09 Partial fill | Demo partial_fill: fills of 0.4 and 0.6. 9 tests in PartialFill, including the same fill read twice, a stale status and a cancel racing a fill | Cumulative fill recorded as 0.4, then 1, and the state moved to FILLED. A fill read twice was counted once, and the remainder was never sent again | Pass |
| T10 Clock offset | Not run | The reference signs no request, so it has no clock check | Not run |
| T11 Emergency stop | Demo kill_switch: switch engaged with one order working. 10 tests in KillSwitchBehavior, including a switch file created from outside, a restart, and a switch that trips while an order is on its way | The working order was canceled and the next request was blocked with kill_switch. The switch survived a restart and failed closed when its file could not be read | Pass |
| T12 Secrets in logs | Not run | There is no key or secret in the reference to search for. The 2 tests in Logging check that every step is one JSON line and that the log survives a crash | Not run |
| T13 Reconciliation mismatch | Tests in RestartRecovery: an orphan order, an order that does not match the journal, an acknowledged order that disappears, and orders from other software | The gateway stayed closed, or the order stayed UNKNOWN, until handled. Orders in another software's ID format were not called orphans. Balances were not compared, because the reference leaves balance reconciliation out | Pass, in part |
| T14 Symbol rules | Not run | Tick size, step size, minimum order value and rounding are left out of the reference on purpose | Not run |
== kill_switch: The kill switch is engaged with one order working
1 request_received request_id=demo-8 side=BUY quantity=1 limit_price=20
2 order_journaled state=SUBMITTING
3 send_attempt attempt=1
4 send_result outcome=OK http_status=200 attempt=1
5 state_changed from_state=SUBMITTING to_state=OPEN
6 kill_switch_engaged reason=operator stopped trading
7 cancel_attempt attempt=1
8 cancel_result outcome=OK http_status=200 attempt=1
9 state_changed from_state=OPEN to_state=CANCELED
10 request_received request_id=demo-9 side=BUY quantity=1 limit_price=20
11 order_blocked reason=kill_switch
result: state=BLOCKED filled=0 reason=kill_switch POST calls=1 orders at exchange=16. Rule checks
Not run. The reference contains no trading rules, so there is nothing to check against a rule document. In a project, two checks confirm that the software applies the customer's written rules as written: W01 replays each worked case from the rule document through the rule module, and P01 traces every instruction in a paper trading period from decision to order to final state. Neither calculates a return, a win rate or an equity curve, because they test the software, not the rules.
7. Defects found
No test failed in this run. A report that shows only passes tells you little, so here is what the tests catch. The reference was reviewed on November 5, 2026, before publication, and its code was fixed in that review. Run against the version committed earlier that day, before those fixes, 13 of the 91 tests fail: 11 failures, and errors in 2 more. Each of the 13 was added or changed after that earlier version. It is not the version in the download, so you cannot repeat this part of the run.
| Test | What the earlier version did |
|---|---|
test_refused_resend_after_false_absence_reads_before_deciding | The first send landed and its answer was lost, two reads said not found, and the second send was refused for another reason. The order was marked REJECTED while it was working at the mock exchange |
test_kill_switch_during_a_retry_finds_and_cancels_an_order_that_landed | Left an order OPEN that had landed during a retry when the switch tripped |
test_order_on_its_way_when_the_switch_trips_is_canceled | Left an order OPEN that was on its way when the switch tripped |
test_same_client_id_with_a_new_exchange_order_id_stays_unknown | Accepted an order whose exchange order ID had changed as OPEN, instead of holding it as UNKNOWN |
test_late_order_that_does_not_match_keeps_the_run_unclean | Did not list a late order with the wrong quantity as unresolved |
test_late_order_the_cancel_cannot_stop_keeps_the_run_unclean | Did not list a late order as unresolved when its cancel failed |
test_send_cut_short_by_an_error_blocks_new_orders_until_reconciled | Accepted a new order after a send was cut short by an error on this machine |
test_log_failure_after_the_journal_write_leaves_the_order_to_reconcile | Did not resolve, on reconciliation, an order whose log write failed after the journal write |
test_fill_with_more_digits_than_decimal_precision_is_kept_exactly | Stored a reported fill of 0.1234567890123456789012345678901 as 0.1234567890123456789012345679 |
test_journal_uses_the_sync_setting_meant_to_survive_a_power_loss | Ran the journal with SQLite's synchronous setting at FULL instead of EXTRA |
test_limits_have_no_default_values | Opened a gateway when no limit values were passed |
test_data_that_is_not_a_mapping_is_refused | Raised TypeError on None and on [{}] instead of refusing the request |
test_answer_nested_too_deep_is_unreadable_not_a_crash | Let a RecursionError escape on a deeply nested JSON answer, instead of treating it as unreadable |
In a project report, each defect also gets an ID, its cause, the fix and the re-test that closed it, and it stays in the report after it is fixed.
8. Timing
Not measured. The mock answers on the same machine and the tests run on an injected clock, so a latency figure from this run would describe the test setup, not order handling. The whole suite took 12.955 s on Windows and 8.269 s on Linux, which says nothing about speed in a live market. A project report states what was measured, on what hardware, against which environment and over how many samples. Timings from one run are observations, not performance promises.
| Measure | Samples | Median | 95th percentile |
|---|---|---|---|
| Instruction received to order acknowledged by the testnet | [count] | [measured] | [measured] |
| Restart to ready, with one unfinished order | [count] | [measured] | [measured] |
9. Limits of this report
- The mock exchange is not a market and not a model of any real platform. A live exchange can fail in ways nobody scripted.
- The reference is teaching code, not production software. It leaves out signing, keys, clock checks, symbol rules, streams, balance reconciliation, alerts and a daily loss limit, so those cases are not run here.
- The crash tests kill a process. They do not cut power or fail a disk.
- The same company wrote the code and the tests. This run shows that the code does what its tests check, not that the tests cover every failure.
- It was run on Windows and on Linux, not on macOS, and not on Python 3.13 or later.
- Tests show how the software behaves, not future results. Trades can lose money.
10. Decision and sign-off
No decision is needed here: there is no client, and nothing goes live. In a project, the proposed next step after this report is a gradual go-live on the customer's own account, with small limits the customer sets, and only with the approval below.
| Role | Name | Date | Decision |
|---|---|---|---|
| Prepared by Fincode | [name] | [date] | [issued] |
| Customer approver | [name] | [date] | [approved for go-live, or changes requested] |
Sample of the format. The results are a real run of Fincode's published teaching reference against its own mock exchange on November 5, 2026, not a client project and not a live market. Fincode is a software company and does not provide investment advice, portfolio management or crypto asset services.
What to look for in any test report
Ask for a sample test report before you hire anyone, and read it with these questions.
Were failures caused on purpose?
Normal orders on a testnet prove little. Look for duplicates, lost responses, dropped connections and a process killed in the middle of an order.
Is there evidence?
A pass should point to something you can check, such as a test name, a log line or a query result, not an adjective.
Does it say what was not run?
Every test setup leaves something out. A report that marks those cases as not run is more useful than one that implies it covered everything.
Did anything fail?
A first run with no failures deserves a question. Defects that were found, fixed and re-tested tell you more than a clean table.
Are there profit figures?
A return or a win rate says nothing about whether the software handles failure. Fincode leaves them out of test reports.
Questions about test reports
Is this a real test report?
It is a real run, but not of a client project. The software is Fincode's published teaching reference, tested against its own mock exchange on one computer. No order reached any exchange, and there is no client behind it.
Can I run these tests myself?
Yes. Download the zip from the reference page, unzip it and run python -m unittest -v and python demo.py with Python 3.11 or newer. Nothing needs installing, and everything stays on 127.0.0.1.
Why are some cases marked as not run?
Because the reference does not contain the code they would test, such as request signing, symbol rules or a streaming connection. Marking them as not run is more honest than leaving them out of the table.
Why are there no backtest results or returns?
Acceptance checks describe how the software behaves. A return, a win rate or a backtest result is not an acceptance criterion, and Fincode does not say whether rules will make money.
Does passing these tests mean software is ready for a live account?
No. A mock exchange is not a live market. In a project, testing continues on the platform's testnet where one exists and in a paper trading account with live market data, and go-live starts with small limits.
Can I see a test report for a client project?
No client project is shown on this site, because Fincode has no client references to show. This run of the published reference is the closest thing you can check for yourself.
Read the other documents
Want your software tested this way?
Send a short brief with the platform you use, what the software should do and your timeline. Fincode is a software company and does not provide investment advice, portfolio management or crypto asset services. Tests show how the software behaves, not future results, and trades can lose money.