Skip to content

Commit 2330916

Browse files
AWeirdDevAWeirdDev
committed
up
1 parent 035e95d commit 2330916

File tree

3 files changed

+30
-142
lines changed

3 files changed

+30
-142
lines changed

‎README.md‎

Lines changed: 24 additions & 137 deletions
Original file line numberDiff line numberDiff line change
@@ -7,155 +7,42 @@
77

88
The fast and strongly-typed Google Flights scraper (API) implemented in Python. Based on Base64-encoded Protobuf string.
99

10-
[Issues](https://github.com/AWeirdDev/flights/issues) • [Discussions](https://github.com/AWeirdDev/flights/discussions)
10+
[**Documentation**](https://aweirddev.github.io/flights) • [Issues](https://github.com/AWeirdDev/flights/issues) • [Discussions](https://github.com/AWeirdDev/flights/discussions)
1111

1212
```haskell
1313
$ pip install fast-flights
1414
```
1515

1616
</div>
1717

18-
## Notes
19-
We're using Protobuf strings to generate the `tfs` query parameter, which stores all the information for a lookup request. We then parse the HTML content and extract the info we need.
20-
21-
Generally speaking, using the `requests` module with naively-inserted `User-Agent` headers to scrape Google websites is a horrible idea since it's too easy to detect on the server-side. I've been blocked once, and it lasted for almost 3 months. If you're looking to be more stable, I recommend using proxies or replace the `requests` module in the source code to `primp`, which is a scraper yet highly optimized for browser impersonation. Since `primp` doesn't come with type annotations, you may create a file named `primp.py` importing the necessary items (`Client`) and constructing a blank class for `Response`, which is not directly importable from `primp`. Type definitions (`.pyi`) for `primp`:
22-
23-
<details>
24-
<summary>Expand <code>primp.pyi</code></summary>
25-
26-
```python
27-
from typing import Dict, Optional, Tuple
28-
29-
class Client:
30-
"""Initializes an HTTP client that can impersonate web browsers.
31-
32-
Args:
33-
auth (tuple, optional): A tuple containing the username and password for basic authentication. Default is None.
34-
auth_bearer (str, optional): Bearer token for authentication. Default is None.
35-
params (dict, optional): Default query parameters to include in all requests. Default is None.
36-
headers (dict, optional): Default headers to send with requests. If `impersonate` is set, this will be ignored.
37-
cookies (dict, optional): - An optional map of cookies to send with requests as the `Cookie` header.
38-
timeout (float, optional): HTTP request timeout in seconds. Default is 30.
39-
cookie_store (bool, optional): Enable a persistent cookie store. Received cookies will be preserved and included
40-
in additional requests. Default is True.
41-
referer (bool, optional): Enable or disable automatic setting of the `Referer` header. Default is True.
42-
proxy (str, optional): Proxy URL for HTTP requests. Example: "socks5://127.0.0.1:9150". Default is None.
43-
impersonate (str, optional): Entity to impersonate. Example: "chrome_124". Default is None.
44-
Chrome: "chrome_100","chrome_101","chrome_104","chrome_105","chrome_106","chrome_107","chrome_108",
45-
"chrome_109","chrome_114","chrome_116","chrome_117","chrome_118","chrome_119","chrome_120",
46-
"chrome_123","chrome_124","chrome_126","chrome_127","chrome_128"
47-
Safari: "safari_ios_16.5","safari_ios_17.2","safari_ios_17.4.1","safari_15.3","safari_15.5","safari_15.6.1",
48-
"safari_16","safari_16.5","safari_17.0","safari_17.2.1","safari_17.4.1","safari_17.5"
49-
OkHttp: "okhttp_3.9","okhttp_3.11","okhttp_3.13","okhttp_3.14","okhttp_4.9","okhttp_4.10","okhttp_5"
50-
Edge: "edge_101","edge_122","edge_127"
51-
follow_redirects (bool, optional): Whether to follow redirects. Default is True.
52-
max_redirects (int, optional): Maximum redirects to follow. Default 20. Applies if `follow_redirects` is True.
53-
verify (bool, optional): Verify SSL certificates. Default is True.
54-
ca_cert_file (str, optional): Path to CA certificate store. Default is None.
55-
http1 (bool, optional): Use only HTTP/1.1. Default is None.
56-
http2 (bool, optional): Use only HTTP/2. Default is None.
57-
58-
"""
59-
60-
def __init__(
61-
self,
62-
auth: Optional[Tuple[str, Optional[str]]] = None,
63-
auth_bearer: Optional[str] = None,
64-
params: Optional[Dict[str, str]] = None,
65-
headers: Optional[Dict[str, str]] = None,
66-
cookies: Optional[Dict[str, str]] = None,
67-
timeout: Optional[float] = 30,
68-
cookie_store: Optional[bool] = True,
69-
referer: Optional[bool] = True,
70-
proxy: Optional[str] = None,
71-
impersonate: Optional[str] = None,
72-
follow_redirects: Optional[bool] = True,
73-
max_redirects: Optional[int] = 20,
74-
verify: Optional[bool] = True,
75-
ca_cert_file: Optional[str] = None,
76-
http1: Optional[bool] = None,
77-
http2: Optional[bool] = None,
78-
): ...
79-
def get(
80-
self,
81-
url: str,
82-
params: Optional[Dict[str, str]] = None,
83-
headers: Optional[Dict[str, str]] = None,
84-
cookies: Optional[Dict[str, str]] = None,
85-
auth: Optional[Tuple[str, Optional[str]]] = None,
86-
auth_bearer: Optional[str] = None,
87-
timeout: Optional[float] = 30,
88-
) -> "Response":
89-
"""Performs a GET request to the specified URL.
90-
91-
Args:
92-
url (str): The URL to which the request will be made.
93-
params (Optional[Dict[str, str]]): A map of query parameters to append to the URL. Default is None.
94-
headers (Optional[Dict[str, str]]): A map of HTTP headers to send with the request. Default is None.
95-
cookies (Optional[Dict[str, str]]): - An optional map of cookies to send with requests as the `Cookie` header.
96-
auth (Optional[Tuple[str, Optional[str]]]): A tuple containing the username and an optional password
97-
for basic authentication. Default is None.
98-
auth_bearer (Optional[str]): A string representing the bearer token for bearer token authentication. Default is None.
99-
timeout (Optional[float]): The timeout for the request in seconds. Default is 30.
100-
101-
"""
102-
103-
class Response:
104-
content: str
105-
cookies: Dict[str, str]
106-
headers: Dict[str, str]
107-
status_code: int
108-
text: str
109-
text_markdown: str
110-
text_plain: str
111-
text_rich: str
112-
url: str
113-
```
114-
115-
</details>
116-
11718
## Basics
118-
**TL;;DR**: To use `fast-flights`, you'll first create a filter (for `?tfs=`) to perform a request.
119-
Then, add `flight_data`, `trip`, `seat`, `passengers`, and (optional) `max_stops` info to use the API directly.
19+
**TL;DR**: To use `fast-flights`, you'll first create a filter (for `?tfs=`) to perform a request.
20+
Then, add `flight_data`, `trip`, `seat`, `passengers` to use the API directly.
12021

12122
```python
122-
from fast_flights import FlightData, Passengers, create_filter, get_flights
23+
from fast_flights import FlightData, Passengers, Result, get_flights
12324

124-
# Create a new filter
125-
filter = create_filter(
25+
result: Result = get_flights(
12626
flight_data=[
127-
# Include more if it's not a one-way trip
128-
FlightData(
129-
date="2024-07-02", # Date of departure
130-
from_airport="TPE",
131-
to_airport="MYJ"
132-
),
133-
# ... include more for round trips
27+
FlightData(date="2025-01-01", from_airport="TPE", to_airport="MYJ")
13428
],
135-
trip="one-way", # Trip (round-trip, one-way)
136-
seat="economy", # Seat (economy, premium-economy, business or first)
137-
passengers=Passengers(
138-
adults=2,
139-
children=1,
140-
infants_in_seat=0,
141-
infants_on_lap=0
142-
),
143-
max_stops=None # or specify number
29+
trip="one-way",
30+
seat="economy",
31+
passengers=Passengers(adults=2, children=1, infants_in_seat=0, infants_on_lap=0),
32+
fetch_mode="fallback",
14433
)
14534

146-
# Get flights with a filter
147-
result = get_flights(filter)
35+
print(result)
14836

14937
# The price is currently... low/typical/high
15038
print("The price is currently", result.current_price)
15139
```
15240

153-
**CLI**: A command-line example script is included as `example.py`. Usage is as follows:
154-
155-
`python3 example.py --origin LAX --destination LGA --depart_date 2025-2-26 --return_date 2025-02-29 --max_stops 1`
41+
**Properties & usage for `Result`**:
15642

157-
**Information**: Display additional information.
15843
```python
44+
result.current_price
45+
15946
# Get the first flight
16047
flight = result.flights[0]
16148

@@ -174,18 +61,18 @@ flight.price
17461

17562
```python
17663
Airport.TAIPEI
177-
|---------------------------------|
178-
| TAIPEI_SONGSHAN_AIRPORT |
179-
| TAPACHULA_INTERNATIONAL_AIRPORT |
180-
| TAMPA_INTERNATIONAL_AIRPORT |
181-
| ... 5 more |
182-
|---------------------------------|
64+
╭─────────────────────────────────╮
65+
│ TAIPEI_SONGSHAN_AIRPORT │
66+
│ TAPACHULA_INTERNATIONAL_AIRPORT │
67+
│ TAMPA_INTERNATIONAL_AIRPORT │
68+
╰─────────────────────────────────╯
18369
```
18470

185-
**Preflights**: We may request to the server twice as sometimes the initial request would not return any results. When this happens, it counts as a preflight agent and we'll send another request to the server as they build data. You can think of this as a "cold start."
186-
18771
## Cookies & consent
188-
The EU region is a bit tricky to solve for now, I'll find workarounds soon.
72+
The EU region is a bit tricky to solve for now, but the fallback support should be able to handle it.
73+
74+
## What's new
75+
- `v2.0` – New (much more succinct) API, fallback support for Playwright serverless functions, and [documentation](https://aweirddev.github.io/flights)!
18976

19077
***
19178

@@ -291,6 +178,6 @@ Yes, please: [github.com/AWeirdDev/flights](https://github.com/AWeirdDev/flights
291178

292179
<div align="center>
293180

294-
(c) AWeirdDev
181+
(c) 2024 AWeirdDev
295182

296183
</div>

‎dist/fast_flights-2.0.tar.gz‎

51.4 KB
Binary file not shown.

‎pyproject.toml‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi"
44

55
[project]
66
name = "fast-flights"
7-
version = "1.2"
7+
version = "2.0"
88
description = "The fast, robust, strongly-typed Google Flights scraper (API) implemented in Python."
99
keywords = ["flights", "google", "google-flights", "scraper", "protobuf", "travel", "trip", "passengers", "airport"]
1010
authors = [
@@ -17,16 +17,17 @@ classifiers = [
1717
"License :: OSI Approved :: MIT License",
1818
"Operating System :: OS Independent",
1919
]
20-
requires-python = ">=3.7"
20+
requires-python = ">=3.8"
2121
dependencies = [
22-
"requests",
22+
"primp",
2323
"protobuf",
2424
"selectolax"
2525
]
2626

2727
[project.urls]
28-
"Homepage" = "https://github.com/AWeirdDev/flights"
29-
"Bug Tracker" = "https://github.com/AWeirdDev/flights/issues"
28+
"Source" = "https://github.com/AWeirdDev/flights"
29+
"Issues" = "https://github.com/AWeirdDev/flights/issues"
30+
"Documentation" = "https://aweirddev.github.io/flights/"
3031

3132
[tool.setuptools]
3233
packages = [

0 commit comments

Comments
 (0)