Webbrowser¶
Warning
The APIs of the streamlink.webbrowser
package are considered unstable. Use at your own risk!
- class streamlink.webbrowser.cdp.client.CDPClient(cdp_connection, nursery, headless)¶
Bases:
object
The public interface around
ChromiumWebbrowser
andCDPConnection
.It launches the Chromium-based web browser, establishes the remote debugging WebSocket connection using the Chrome Devtools Protocol, and provides the
session()
method for creating a newCDPClientSession
that is tied to an empty new browser tab.CDPClientSession
provides a high-level API for navigating websites, intercepting network requests and responses, as well as evaluating JavaScript expressions and retrieving async results.Don't instantiate this class yourself, use the
CDPClient.launch()
async context manager classmethod.For low-level Chrome Devtools Protocol interfaces, please see Streamlink's automatically generated
streamlink.webbrowser.cdp.devtools
package, but be aware that only a subset of the available domains is supported.- Parameters:
cdp_connection (CDPConnection)
nursery (Nursery)
headless (bool)
- classmethod launch(session, runner, executable=None, timeout=None, cdp_host=None, cdp_port=None, cdp_timeout=None, headless=None)¶
Start a new
trio
runloop and do the following things:Launch the Chromium-based web browser using the provided parameters or respective session options
Initialize a new
CDPConnection
and connect to the browser's remote debugging interfaceCreate a new
CDPClient
instanceExecute the async runner callback with the
CDPClient
instance as only argument
If the
webbrowser
session option is set toFalse
, then aCDPError
will be raised.Example:
async def fake_response(client_session: CDPClientSession, request: devtools.fetch.RequestPaused): if request.response_status_code is not None and 300 <= request.response_status_code < 400: await client_session.continue_request(request) else: async with client_session.alter_request(request) as cmproxy: cmproxy.body = "<!doctype html><html><body>foo</body></html>" async def my_app_logic(client: CDPClient): async with client.session() as client_session: client_session.add_request_handler(fake_response, "*") async with client_session.navigate("https://google.com") as frame_id: await client_session.loaded(frame_id) return await client_session.evaluate("document.body.innerText") assert CDPClient.launch(session, my_app_logic) == "foo"
- Parameters:
session (Streamlink) -- The Streamlink session object
runner (Callable[[Self], Coroutine]) -- An async client callback function which receives the
CDPClient
instance as only parameter.executable (str | None) -- Optional path to the Chromium-based web browser executable. If unset, falls back to the
webbrowser-executable
session option. Otherwise, it'll be looked up according to the rules of theChromiumBrowser
implementation.timeout (float | None) -- Optional global timeout value, including web browser launch time. If unset, falls back to the
webbrowser-timeout
session option.cdp_host (str | None) -- Optional remote debugging host. If unset, falls back to the
webbrowser-cdp-host
session option. Otherwise,127.0.0.1
will be used.cdp_port (int | None) -- Optional remote debugging port. If unset, falls back to the
webbrowser-cdp-port
session option. Otherwise, a random free port will be chosen.cdp_timeout (float | None) -- Optional CDP command timeout value. If unset, falls back to the
webbrowser-cdp-timeout
session option.headless (bool | None) -- Optional boolean flag whether to launch the web browser in headless mode or not. If unset, falls back to the
webbrowser-headless
session option.
- Return type:
Any
- session(fail_unhandled_requests=False, max_buffer_size=None)¶
Create a new CDP session on an empty target (browser tab).
- Parameters:
fail_unhandled_requests (bool) -- Whether network requests which are not matched by any request handlers should fail.
max_buffer_size (int | None) -- Optional size of the send/receive memory channel for paused HTTP requests/responses.
- Return type:
AsyncGenerator[CDPClientSession, None]
- class streamlink.webbrowser.cdp.client.CDPClientSession(cdp_client, cdp_session, fail_unhandled_requests=False, max_buffer_size=None)¶
Bases:
object
High-level API for navigating websites, intercepting network requests/responses, and for evaluating async JavaScript expressions.
Don't instantiate this class yourself, use the
CDPClient.session()
async contextmanager.- Parameters:
cdp_client (CDPClient)
cdp_session (CDPSession)
fail_unhandled_requests (bool)
max_buffer_size (int | None)
- add_request_handler(async_handler, url_pattern='*', on_request=False)¶
- Parameters:
async_handler (Callable[[CDPClientSession, RequestPaused], Awaitable]) -- An async request handler which must call
continue_request()
,fail_request()
,fulfill_request()
oralter_request()
, or the next matching request handler will be run. If no matching request handler was found or if no matching one called one of the just mentioned methods, then the request will be continued if the session was initialized withfail_unhandled_requests=False
, otherwise it will be blocked.url_pattern (str) -- An optional URL wildcard string which defaults to
"*"
. Only matching URLs will causeFetch.requestPraused
events to be emitted over the CDP connection. The async request handler will be called on each matching URL unless another request handler has already handled the request (see description above).on_request (bool) -- Whether to intercept the network request or the network response.
- alter_request(request, response_code=200, response_headers=None)¶
Async context manager wrapper around
fulfill_request()
which retrieves the response body, so it can be altered. The status code and headers can be altered in the method call directly, or by setting the respective parameters on the context manager's proxy object.- Parameters:
request (RequestPaused)
response_code (int)
response_headers (Mapping[str, str] | None)
- Return type:
AsyncGenerator[CMRequestProxy, None]
- async continue_request(request, url=None, method=None, post_data=None, headers=None)¶
Continue a request and optionally override the request method, URL, POST data or request headers.
- Parameters:
request (RequestPaused)
url (str | None)
method (str | None)
post_data (str | None)
headers (Mapping[str, str] | None)
- async evaluate(expression, await_promise=True, timeout=None)¶
Evaluate an optionally async JavaScript expression and return its result.
- Parameters:
expression (str) -- The JavaScript expression.
await_promise (bool) -- Whether to await a returned
Promise()
object.timeout (float | None) -- Optional timeout override value. Uses the session's single CDP command timeout value by default, which may be too short depending on the script execution time.
- Raises:
CDPError -- On evaluation error or if the result is a subtype of
window.Error()
.- Returns:
Only JS-primitive result values are supported, e.g. strings or numbers. Other kinds of return values must be serialized, e.g. via
JSON.stringify()
.- Return type:
Any
- async fail_request(request, error_reason=None)¶
Let a request fail, with an optional error reason which defaults to
BlockedByClient
.- Parameters:
request (RequestPaused)
error_reason (str | None)
- async fulfill_request(request, response_code=200, response_headers=None, body=None)¶
Fulfill a response and override its status code, headers and body.
- Parameters:
request (RequestPaused)
response_code (int)
response_headers (Mapping[str, str] | None)
body (str | None)
- Return type:
None
- async loaded(frame_id)¶
Wait for the navigated page to finish loading.
- Parameters:
frame_id (FrameId)
Async context manager for opening the URL with an optional referrer and starting the optional interception of network requests and responses. If the target gets detached from the session, e.g. by closing the tab, then the whole CDP connection gets terminated, including all other concurrent sessions. Doesn't wait for the request to finish loading. See
loaded()
.- Parameters:
url (str) -- The URL.
referrer (str | None) -- An optional referrer.
- Returns:
Yields the
FrameID
that can be passed to theloaded()
call.- Return type:
AsyncGenerator[FrameId, None]
- class streamlink.webbrowser.cdp.client.CMRequestProxy(body: str, response_code: int, response_headers: Mapping[str, str] | None)¶
Bases:
object
- Parameters:
body (str)
response_code (int)
response_headers (Mapping[str, str] | None)
- class streamlink.webbrowser.cdp.connection.CDPBase(websocket, target_id=None, session_id=None, cmd_timeout=2)¶
Bases:
object
Low-level base class for Chrome Devtools Protocol connection & session management.
It provides methods for sending CDP commands and receiving their responses, as well as for listening to CDP events.
Both CDP commands and events can be sent and received in a global context and in a session context.
The Chrome Devtools Protocol is documented at https://chromedevtools.github.io/devtools-protocol/
- Parameters:
websocket (WebSocketConnection)
target_id (Optional[TargetID])
session_id (Optional[SessionID])
cmd_timeout (float)
- listen(event, max_buffer_size=None)¶
Listen to a CDP event and return a new
CDPEventListener
instance.- Parameters:
event (Type[TEvent]) -- See the
streamlink.webbrowser.cdp.devtools
package for the available events. For events to be sent over the CDP connection, a specific domain needs to be enabled first.max_buffer_size (int | None) -- The buffer size of the
trio
memory channel.
- Returns:
- Return type:
CDPEventListener[TEvent]
- async send(cmd, timeout=None)¶
Send a specific CDP command and await its response.
- Parameters:
cmd (Generator[Dict[str, Any], Dict[str, Any], TCmdResponse]) -- See the
streamlink.webbrowser.cdp.devtools
package for the available commands.timeout (float | None) -- Override of the max amount of time a response can take. Uses the class's default value otherwise. This override is mostly only relevant for awaiting JS code evaluations.
- Returns:
The return value depends on the used command.
- Return type:
TCmdResponse
- class streamlink.webbrowser.cdp.connection.CDPConnection(websocket, cmd_timeout)¶
Bases:
CDPBase
,AsyncResource
Don't instantiate this class yourself, see its
create()
classmethod.- Parameters:
websocket (WebSocketConnection)
cmd_timeout (float)
- async aclose()¶
Close the websocket connection, close all memory channels and clean up all opened sessions.
- Return type:
None
- classmethod create(url, timeout=None)¶
Establish a new CDP connection to the Chromium-based web browser's remote debugging interface.
- Parameters:
url (str) -- The websocket address
timeout (float | None) -- The max amount of time a single CDP command response can take.
- Returns:
- Return type:
AsyncGenerator[Self, None]
- async new_target(url='')¶
Create a new target (browser tab) and return a new
CDPSession
instance.- Parameters:
url (str) -- Optional URL. Leave empty for a blank target (preferred for proper navigation handling).
- Returns:
- Return type:
- class streamlink.webbrowser.cdp.connection.CDPSession(websocket, target_id=None, session_id=None, cmd_timeout=2)¶
Bases:
CDPBase
- Parameters:
websocket (WebSocketConnection)
target_id (Optional[TargetID])
session_id (Optional[SessionID])
cmd_timeout (float)
- class streamlink.webbrowser.cdp.connection.CDPEventListener(event_channels, event, max_buffer_size=None)¶
Bases:
Generic
[TEvent
]Instances of this class are returned by
CDPBase.listen()
.The return types of each of its methods depend on the event type.
Can be used as an async for-loop which indefinitely waits for events to be emitted, or can be used as an async context manager which yields the event and closes the listener when leaving the context manager.
Example:
async def listen(cdp_session: CDPSession): async for request in cdp_session.listen(devtools.fetch.RequestPaused): ... async def listen_once(cdp_session: CDPSession): async with cdp_session.listen(devtools.fetch.RequestPaused) as request: ...
- Parameters:
event_channels (TEventChannels)
event (Type[TEvent])
max_buffer_size (Optional[int])
- async receive()¶
Await a single event without closing the listener's memory channel.
- Return type:
TEvent
- class streamlink.webbrowser.webbrowser.Webbrowser(executable=None)¶
Bases:
object
- Parameters:
executable (str | None)
- class streamlink.webbrowser.chromium.ChromiumWebbrowser(*args, host=None, port=None, **kwargs)¶
Bases:
Webbrowser
- Parameters:
host (str | None)
port (int | None)