Skip to content

Commit 1fc037f

Browse files
authored
docs: Document URL field normalization in typed models (#1005)
URL fields on the returned models are `pydantic.AnyUrl`, which normalizes the value during validation, so the string read back can differ from the one the API sent. The trailing slash is the most visible case; normalization also lowercases the host, drops default ports, punycodes internationalized hosts, and percent-encodes unsafe characters. The behavior is intended, it just wasn't documented. - `docs/02_concepts/12_typed_models.mdx` gets a "URL fields" section covering the normalization cases, comparing URLs as `AnyUrl` rather than raw strings, and building longer URLs with `urljoin`. - `docs/04_upgrading/upgrading_to_v3.mdx` gets a "URL fields are normalized" section framing it as the v2 to v3 change, with the affected field names. Closes #999 *✍️ Drafted by Claude Code*
1 parent 37730e8 commit 1fc037f

2 files changed

Lines changed: 65 additions & 0 deletions

File tree

docs/02_concepts/12_typed_models.mdx

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,50 @@ Every method that returns a structured payload returns a Pydantic model. Fields
3636

3737
Date strings are automatically parsed into timezone-aware `datetime.datetime` objects, enums into `Literal` aliases, and nested objects into their own typed models, so you can compose attribute access without manual conversion.
3838

39+
## URL fields
40+
41+
Fields holding a URL, such as `Run.container_url` or `Dataset.console_url`, are typed as [`AnyUrl`](https://docs.pydantic.dev/latest/api/networks/#pydantic.networks.AnyUrl). Their values are validated and normalized, so the string you read back can differ from the one the API sent. Both forms denote the same URL under [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986#section-6.2.3), they're just different strings.
42+
43+
```python
44+
from pydantic import AnyUrl
45+
46+
# An empty path becomes '/'.
47+
str(AnyUrl('https://abc123.runs.apify.net')) # 'https://abc123.runs.apify.net/'
48+
49+
# The host is lowercased.
50+
str(AnyUrl('https://EXAMPLE.com/Path')) # 'https://example.com/Path'
51+
52+
# A default port is dropped.
53+
str(AnyUrl('https://example.com:443/path')) # 'https://example.com/path'
54+
55+
# An internationalized host is punycoded.
56+
str(AnyUrl('https://www.žluty.cz')) # 'https://www.xn--luty-kbb.cz/'
57+
58+
# Unsafe characters are percent-encoded.
59+
str(AnyUrl('https://example.com/a b')) # 'https://example.com/a%20b'
60+
```
61+
62+
Because of the normalization, compare URLs in their normalized form instead of as raw strings. `AnyUrl` compares the parsed URL, not the input text.
63+
64+
```python
65+
run = client.run('my-run-id').get()
66+
stored_url = 'https://abc123.runs.apify.net'
67+
68+
# Wrong, compares a raw string with a normalized one.
69+
stored_url == str(run.container_url) # False
70+
71+
# Right, both sides are normalized before the comparison.
72+
AnyUrl(stored_url) == run.container_url # True
73+
```
74+
75+
To build a longer URL out of a URL field, use [`urljoin`](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urljoin) rather than string concatenation. It gives the same result whether or not the base ends with a slash.
76+
77+
```python
78+
from urllib.parse import urljoin
79+
80+
urljoin(str(run.container_url), 'status') # 'https://abc123.runs.apify.net/status'
81+
```
82+
3983
## Providing structured input
4084

4185
A Pydantic model returned from one client call can be passed directly into any other method that accepts the same shape — useful for round-trip flows where you read a resource, tweak it, and write it back.

docs/04_upgrading/upgrading_to_v3.mdx

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,27 @@ Two endpoints still return plain types because their payloads are user-defined:
3030

3131
On the input side, methods now also accept Pydantic models in addition to dicts. Plain dicts continue to work — keys may use either snake_case or camelCase, and each input shape has a matching [`TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict) so editors and type checkers can validate the keys. See [Typed models](/api/client/python/docs/concepts/typed-models) for details.
3232

33+
## URL fields are normalized
34+
35+
URL fields on the returned models are typed as [`AnyUrl`](https://docs.pydantic.dev/latest/api/networks/#pydantic.networks.AnyUrl), which normalizes the value during validation. In v2 the client handed back the raw string from the API. In v3 the string can differ, most visibly by an added trailing slash. Normalization also lowercases the host, drops default ports, punycodes internationalized hosts, and percent-encodes unsafe characters.
36+
37+
Code that compares a stored URL with a URL field has to compare normalized values.
38+
39+
```python
40+
from pydantic import AnyUrl
41+
42+
run = client.run('my-run-id').get()
43+
stored_url = 'https://abc123.runs.apify.net'
44+
45+
# Before, the raw string matched the API response.
46+
stored_url == str(run.container_url) # False in v3
47+
48+
# After, normalize both sides before comparing.
49+
AnyUrl(stored_url) == run.container_url # True
50+
```
51+
52+
The affected fields are `container_url`, `console_url`, `standby_url`, `request_url`, `url`, `picture_url`, `user_picture_url`, `website_url`, and the `*_public_url` fields on storage models. To build longer URLs out of them, use [`urljoin`](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urljoin) instead of string concatenation. For details, see [Typed models](/api/client/python/docs/concepts/typed-models#url-fields).
53+
3354
## Tiered timeouts
3455

3556
The single global timeout has been replaced by four tiers — `short` (5 s), `medium` (30 s), `long` (360 s), and `no_timeout`. Each method picks an appropriate default. Override per call, or change tier defaults on the constructor:

0 commit comments

Comments
 (0)