How to test trading bots: a plan before going live

Short answer: To test a trading bot, work in layers: unit tests for rounding, contract tests on recorded API responses, a mock exchange that injects timeouts, 429s, 5xx errors, partial fills and duplicate messages, and restart and replay tests. Then run a testnet or demo, paper trading, small live size and a kill switch drill, each with written pass conditions. Tests show how the software behaves, not whether the rules make money; a testnet result is no sign of a live result.

Pages ranking for this question on 5 November 2026 mostly cover paper trading, backtesting or a platform's test button, which ask whether rules would have worked (see backtesting pitfalls); this plan asks whether the software behaves when something fails.

Examples contain no strategy or parameter and recommend no exchange. The harness ran on synthetic data on 5 November 2026 with Python 3.11.3 and that day's latest PyPI releases: pytest 9.2.0 (MIT, released 27 October 2026), Hypothesis 6.170.14 (MPL-2.0, 30 October) and CCXT 4.5.92 (MIT, 4 November), each with GitHub pushes within three days. Rules depend on jurisdiction and each venue's terms. Article 99/A of Capital Markets Law No. 6362 treats activity by foreign crypto-asset platforms directed at Turkish residents as unauthorized.

Trading bot testing: what each layer proves and misses

No single environment proves a bot is ready; each answers a narrower question than it seems.

Trading bot testing with pytest: a mock exchange that fails on purpose

A small harness injects each fault deliberately, and is only worth trusting once it has failed on code with known bugs.

pytest's parametrize (opens in a new tab) runs one test per fault; Hypothesis (opens in a new tab), whose tests "should pass for all inputs in whatever range you describe", generates duplicated and out-of-order fills. The fake venue raises CCXT's exception classes; the last two tests run a real ccxt.binance over a fake network with placeholder keys. Save the public exchangeInfo response as recorded/exchangeInfo.json; step and tick are from Binance Spot's response at 19:14 UTC on 5 November 2026.

"""Synthetic data only: no network, no keys, no trading rule."""
import json, os
from decimal import Decimal as D, ROUND_DOWN
import ccxt, pytest, requests
from hypothesis import given, strategies as st

BUGGY = os.environ.get("BUGGY") == "1"       # run the same tests against common bugs
STEP, TICK = D("0.00001"), D("0.01")         # from exchangeInfo, recorded below
RECORDED = os.path.join(os.path.dirname(__file__), "recorded", "exchangeInfo.json")

def round_order(qty, price):
    if BUGGY:
        return round(qty, 5), round(price, 2)
    try:
        q, p = qty.quantize(STEP, ROUND_DOWN), price.quantize(TICK, ROUND_DOWN)
        ok = q > 0 and p > 0                 # zero after rounding
    except ArithmeticError:                  # NaN, Infinity, 1e400
        ok = False
    if not ok:
        raise ValueError(f"refusing {qty} at {price}")
    return q, p

class FakeVenue:
    """One scripted fault per place() call."""
    def __init__(self, *faults):
        self.faults, self.orders, self.calls = list(faults), {}, 0

    def place(self, cid, qty):
        self.calls += 1
        fault = self.faults.pop(0) if self.faults else None
        if fault == "429":
            raise ccxt.RateLimitExceeded("429")
        if fault == "timeout before":
            raise ccxt.RequestTimeout("no reply")
        self.orders[cid] = qty               # the venue has the order from here on
        if fault == "timeout after":
            raise ccxt.RequestTimeout("no reply")
        if fault == "-1006 after":
            raise ccxt.OperationFailed("execution status unknown")

def send(venue, cid, qty):
    """A doubtful outcome is looked up, never resent."""
    if BUGGY:                                # retry network errors with a fresh ID
        for attempt in range(3):
            try:
                venue.place(f"{cid}-{attempt}", qty)
                return "SENT"
            except ccxt.NetworkError:
                pass
        return "UNKNOWN"
    if not cid:                              # "" matches any order sent without an ID
        raise ValueError("client order ID required")
    try:
        venue.place(cid, qty)
        return "SENT"
    except (ccxt.RateLimitExceeded, ccxt.DDoSProtection):
        return "BACK OFF"                    # no sends until Retry-After has passed
    except ccxt.OperationFailed:             # timeout, 5xx, -1006, -1007: outcome unknown
        return "SENT" if cid in venue.orders else "UNKNOWN"

@pytest.mark.parametrize("fault, status, orders", [
    (None, "SENT", 1), ("timeout before", "UNKNOWN", 0), ("timeout after", "SENT", 1),
    ("-1006 after", "SENT", 1), ("429", "BACK OFF", 0)])
def test_one_order_whatever_fails(fault, status, orders):
    venue = FakeVenue(fault)
    assert send(venue, "bot-1", D(1)) == status
    assert len(venue.orders) == orders and venue.calls == 1

def test_empty_client_id_is_refused():
    venue = FakeVenue("timeout before")
    venue.orders[""] = D(5)                  # an older order without an ID
    with pytest.raises(ValueError):
        send(venue, "", D(1))

def apply_report(filled, cum_qty):          # reports carry cumulative quantity
    return cum_qty if BUGGY else max(filled, cum_qty)

@given(st.lists(st.integers(1, 1000), min_size=1, max_size=20), st.randoms())
def test_partial_fills_survive_duplicates_and_reordering(fills, rnd):
    reports = [D(sum(fills[:i + 1])) for i in range(len(fills))]
    delivered = reports + rnd.choices(reports, k=3)    # duplicated messages
    rnd.shuffle(delivered)                             # out-of-order events
    filled = D(0)
    for report in delivered:
        filled = apply_report(filled, report)
    assert filled == sum(fills)

@given(st.decimals("0.001", "1000", places=8), st.decimals("1", "100000", places=4))
def test_rounding_stays_inside_lot_and_tick(qty, price):
    q, p = round_order(qty, price)
    assert q <= qty and p <= price           # rounding up could overspend
    assert q % STEP == 0 and p % TICK == 0

@pytest.mark.parametrize("bad", ["NaN", "Infinity", "1e400", "1e-400"])
def test_bad_numbers_never_become_orders(bad):
    with pytest.raises(ValueError):
        round_order(D(bad), D(100))

def test_contract_recorded_exchange_info():
    with open(RECORDED, encoding="utf-8") as f:
        info = json.load(f)
    assert "symbols" in info, info.get("msg")          # a saved error body
    [symbol] = [s for s in info["symbols"] if s["symbol"] == "BTCUSDT"]
    [lot] = [x for x in symbol["filters"] if x["filterType"] == "LOT_SIZE"]
    [tick] = [x for x in symbol["filters"] if x["filterType"] == "PRICE_FILTER"]
    assert D(lot["stepSize"]) == STEP and D(tick["tickSize"]) == TICK

class FakeHTTP(requests.Session):
    def __init__(self, *replies):
        super().__init__()
        self.replies, self.posts = list(replies), 0

    def request(self, method, url, **kw):
        status, body = 200, "[]"
        if method == "POST":
            self.posts += 1
            status, body = self.replies.pop(0)
        elif "exchangeInfo" in url:
            body = open(RECORDED, encoding="utf-8").read()
        reply = requests.Response()
        reply.status_code, reply._content, reply.url, reply.reason = status, body.encode(), url, ""
        return reply

def order_via_ccxt(http, **options):
    ex = ccxt.binance({"apiKey": "fake", "secret": "fake",   # never leave the process
                       "options": {"fetchMarkets": {"types": ["spot"]}, **options}})
    ex.session = http
    return ex.create_order("BTC/USDT", "limit", "buy", 0.001, 50000, {"newClientOrderId": "bot-1"})

def test_ccxt_raises_minus_1006_outside_network_error():
    http = FakeHTTP((500, '{"code":-1006,"msg":"Execution status unknown"}'))
    with pytest.raises(ccxt.OperationFailed) as raised:
        order_via_ccxt(http)
    assert not isinstance(raised.value, ccxt.NetworkError) and http.posts == 1

def test_ccxt_built_in_retry_resends_the_order():
    http = FakeHTTP((500, ""), (500, ""), (500, ""))
    with pytest.raises(ccxt.ExchangeNotAvailable):
        order_via_ccxt(http, maxRetriesOnFailure=2)
    assert http.posts == 3                   # one call, three identical orders

Run it twice, then with common bugs switched on:

$ pytest -q
15 passed in 4.50s
$ BUGGY=1 pytest -q --tb=no
11 failed, 4 passed in 4.47s

Which CCXT errors are safe to retry?

Reads are safe to retry; for an order, CCXT's exception class does not say whether the order exists, so look it up by client ID before sending again.

The CCXT Manual (opens in a new tab), read 5 November 2026, says after a RequestTimeout "for fetching requests it is safe to retry the call"; a timed-out createOrder() should be checked with fetchOrders(), fetchOpenOrders() or fetchClosedOrders(), then fetchBalance(). Binance documents a 5XX as execution status UNKNOWN. The same fake network, with more replies and endpoints, gave:

So catch OperationFailed, not just NetworkError; on Options, where the manual describes ExchangeError as mostly requests that cannot succeed, check the venue's code too. The manual's text and diagram place RateLimitExceeded and BadResponse differently from the 4.5.92 source, so test the classes you catch. The maxRetriesOnFailure option, 0 by default, retries any OperationFailed: one create_order call sent the same order three times. Keep it at 0 for orders and pin the CCXT version; see client order IDs.

Restart, reconcile and replay tests

Kill the process at each line that touches the venue, restart from durable state, and check for duplicate orders and the filled quantity; trading bot architecture has such a crash test, with a partial fill during the downtime.

For replay, feed recorded market data and order events back with an injected clock and assert identical decisions across two runs; add gaps, repeats and a stale feed, and check that the bot stops opening positions when data stops.

Testnets and demo modes: what to test there

Testnets exercise signing, permissions and real rejections, but each differs from production in ways to test.

• Binance Spot Testnet: its FAQ, read 5 November 2026, says it resets "approximately once per month" without notice and calls order rate limits "generally the same" as the Spot API, yet exchangeInfo that day returned 50 orders per 10 seconds against production's 100.
• OKX: Demo Trading shares the production REST host and needs the header x-simulated-trading: 1; test that it is never missing.
• CCXT: in 4.5.92, set_sandbox_mode(True) raised TypeError for coinbase, kraken and mexc (4.5.10 raised NotSupported); if that is swallowed, the object stays on production.

No test-environment order was placed for this page, as that needs a key; the paper trading API guide compares environments.

Trading bot simulator: trying a bot without real money

A simulator or paper mode runs logic on prices without risking funds, but it fills orders by its own rules, so it cannot show what a real order would get.

A search for a free or online trading bot simulator on 5 November 2026 returned vendors' demo modes, browser games, extensions and app listings, none reviewed here. Check each paper mode for these gaps:

• Slippage: fills at the quote, while a real order walks the book.
• Queue position: a touched limit can count as filled.
• Impact: no liquidity is taken.
• Failures: rejections, rate limits and timeouts rarely occur.

From small live size to a kill switch drill

Go live at the smallest size the venue allows, with limits on, and raise size only after your written criteria have held; the drill proves you can stop it.

1. Run the build and config that passed testing, with a read-and-trade key and no withdrawal right; see API key security.
2. Reconcile orders, fills and balances against the venue on a schedule, and stop on any unexplained difference.
3. Pull the kill switch on purpose: new orders blocked, working orders canceled, position reported, alert received, a person resets it. Time it, write it down, and repeat with the strategy process hung.

Limit design is in trading bot risk management; webhook stages, with TradingView's terms on automated use, are in the TradingView webhook guide.

Written acceptance criteria: behavior, never profit

Accept the software against statements about what it does, because a profit figure can be tuned to pass and says nothing about failures.

A trading bot code review can run such a sheet; naming any venue here does not mean Fincode integrates it or has built on it. Passing shows how the software behaves, not future results; trades can lose money.

Frequently asked questions

Is there a free trading bot simulator online?

Vendors offer demo modes and some apps are games, none reviewed here; they test the vendor's engine, not your code. A pytest harness and a public testnet cost nothing.

Is it safe to retry a CCXT order after a timeout?

Not blindly. The CCXT Manual says to check open, closed or all orders, then the balance, since the order may have been placed.

Does a passing testnet run mean the bot will work live?

No. Live liquidity, fills and latency differ, and a testnet result is no sign of a live result.

How long should a bot run on testnet before going live?

No accepted number exists; set criteria for events seen, such as restarts, rejections and reconciliations.

Sources

• Binance Spot Test Network: F.A.Q. (resets, rate limits) (opens in a new tab)
• Binance Spot API documentation: REST API (HTTP return codes, 5XX execution status unknown) (opens in a new tab)
• Binance Spot exchangeInfo for BTCUSDT, live, via the market data host (opens in a new tab)
• Binance Spot Testnet exchangeInfo for BTCUSDT, live (opens in a new tab)
• OKX API v5 documentation: Demo Trading Services (opens in a new tab)
• pytest documentation: How to parametrize fixtures and test functions (opens in a new tab)
• Hypothesis documentation (opens in a new tab)
• CCXT Manual: Error Handling (retry mechanism, exception hierarchy, RequestTimeout) (opens in a new tab)
• CCXT source: python/ccxt/base/exchange.py (fetch2 retries, HTTP status map, set_sandbox_mode) (opens in a new tab)
• CCXT source: python/ccxt/binance.py (Binance error code map) (opens in a new tab)
• PyPI: pytest (release history, license) (opens in a new tab)
• PyPI: hypothesis (release history, license) (opens in a new tab)
• PyPI: ccxt (release history, license) (opens in a new tab)
• Capital Markets Law No. 6362, consolidated text, Article 99/A (Turkish) (opens in a new tab)

This guide is general information about software and markets, not investment, legal or tax advice. Automated trading can lose money, past or backtested performance does not guarantee future results, and the rules that apply depend on your jurisdiction and your exchange's terms. Sources, checks and corrections: see the editorial policy.