Models

class netlib.http.Request(*args, **kwargs)[source]

An HTTP request.

Data

first_line_format

HTTP request form as defined in RFC7230.

origin-form and asterisk-form are subsumed as “relative”.

method

HTTP request method, e.g. “GET”.

scheme

HTTP request scheme, which should be “http” or “https”.

host

Target host. This may be parsed from the raw request (e.g. from a GET http://example.com/ HTTP/1.1 request line) or inferred from the proxy mode (e.g. an IP in transparent mode).

Setting the host attribute also updates the host header, if present.

port

Target port

path

HTTP request path, e.g. “/index.html”. Guaranteed to start with a slash.

http_version

Version string, e.g. “HTTP/1.1”

headers

Message headers object

Returns:netlib.http.Headers
content

The raw (encoded) HTTP message body

See also: text

timestamp_start

First byte timestamp

timestamp_end

Last byte timestamp

Computed Properties and Convenience Methods

text

The decoded HTTP message body. Decoded contents are not cached, so accessing this attribute repeatedly is relatively expensive.

Note

This is not implemented yet.

See also: content, decoded

url

The URL string, constructed from the request’s URL components

pretty_host

Similar to host, but using the Host headers as an additional preferred data source. This is useful in transparent mode where host is only an IP address, but may not reflect the actual destination as the Host header could be spoofed.

pretty_url

Like url, but using pretty_host instead of host.

query

The request query string as an ODict object. None, if there is no query.

cookies

The request cookies. An empty ODict object if the cookie monster ate them all.

path_components

The URL’s path components as a list of strings. Components are unquoted.

anticache()[source]

Modifies this request to remove headers that might produce a cached response. That is, we remove ETags and If-Modified-Since headers.

anticomp()[source]

Modifies this request to remove headers that will compress the resource’s data.

constrain_encoding()[source]

Limits the permissible Accept-Encoding values, based on what we can decode appropriately.

urlencoded_form

The URL-encoded form data as an ODict object. None if there is no data or the content-type indicates non-form data.

multipart_form

The multipart form data as an ODict object. None if there is no data or the content-type indicates non-form data.

class netlib.http.Response(*args, **kwargs)[source]

An HTTP response.

Data

http_version

Version string, e.g. “HTTP/1.1”

status_code

HTTP Status Code, e.g. 200.

reason

HTTP Reason Phrase, e.g. “Not Found”. This is always None for HTTP2 requests, because HTTP2 responses do not contain a reason phrase.

headers

Message headers object

Returns:netlib.http.Headers
content

The raw (encoded) HTTP message body

See also: text

timestamp_start

First byte timestamp

timestamp_end

Last byte timestamp

Computed Properties and Convenience Methods

text

The decoded HTTP message body. Decoded contents are not cached, so accessing this attribute repeatedly is relatively expensive.

Note

This is not implemented yet.

See also: content, decoded

cookies

Get the contents of all Set-Cookie headers.

A possibly empty ODict, where keys are cookie name strings, and values are [value, attr] lists. Value is a string, and attr is an ODictCaseless containing cookie attributes. Within attrs, unary attributes (e.g. HTTPOnly) are indicated by a Null value.

class netlib.http.Headers(fields=None, **headers)[source]

Header class which allows both convenient access to individual headers as well as direct access to the underlying raw data. Provides a full dictionary interface.

Example:

# Create headers with keyword arguments
>>> h = Headers(host="example.com", content_type="application/xml")

# Headers mostly behave like a normal dict.
>>> h["Host"]
"example.com"

# HTTP Headers are case insensitive
>>> h["host"]
"example.com"

# Headers can also be creatd from a list of raw (header_name, header_value) byte tuples
>>> h = Headers([
    [b"Host",b"example.com"],
    [b"Accept",b"text/html"],
    [b"accept",b"application/xml"]
])

# Multiple headers are folded into a single header as per RFC7230
>>> h["Accept"]
"text/html, application/xml"

# Setting a header removes all existing headers with the same name.
>>> h["Accept"] = "application/text"
>>> h["Accept"]
"application/text"

# bytes(h) returns a HTTP1 header block.
>>> print(bytes(h))
Host: example.com
Accept: application/text

# For full control, the raw header fields can be accessed
>>> h.fields
Caveats:
For use with the “Set-Cookie” header, see get_all().
__init__(fields=None, **headers)[source]
Parameters:
  • fields – (optional) list of (name, value) header byte tuples, e.g. [(b"Host", b"example.com")]. All names and values must be bytes.
  • **headers – Additional headers to set. Will overwrite existing values from fields. For convenience, underscores in header names will be transformed to dashes - this behaviour does not extend to other methods. If **headers contains multiple keys that have equal .lower() s, the behavior is undefined.
get_all(name)[source]

Like get(), but does not fold multiple headers into a single one. This is useful for Set-Cookie headers, which do not support folding.

See also: https://tools.ietf.org/html/rfc7230#section-3.2.2

set_all(name, values)[source]

Explicitly set multiple headers for the given key. See: get_all()

class netlib.http.decoded(message)[source]

A context manager that decodes a request or response, and then re-encodes it with the same encoding after execution of the block.

Example:

with decoded(request):
    request.content = request.content.replace("foo", "bar")
class libmproxy.models.HTTPFlow(client_conn, server_conn, live=None)[source]

Bases: libmproxy.models.flow.Flow

A HTTPFlow is a collection of objects representing a single HTTP transaction.

request

HTTPRequest object

response

HTTPResponse object

error

Error object

server_conn

ServerConnection object

client_conn

ClientConnection object

intercepted

Is this flow currently being intercepted?

live

Does this flow have a live client connection?

Note that it’s possible for a Flow to have both a response and an error object. This might happen, for instance, when a response was received from the server, but there was an error sending it back to the client.

match(f)[source]

Match this flow against a compiled filter expression. Returns True if matched, False if not.

If f is a string, it will be compiled as a filter expression. If the expression is invalid, ValueError is raised.

replace(pattern, repl, *args, **kwargs)[source]

Replaces a regular expression pattern with repl in both request and response of the flow. Encoded content will be decoded before replacement, and re-encoded afterwards.

Returns the number of replacements made.

class libmproxy.models.Error(msg, timestamp=None)[source]

Bases: libmproxy.stateobject.StateObject

An Error.

This is distinct from an protocol error response (say, a HTTP code 500), which is represented by a normal HTTPResponse object. This class is responsible for indicating errors that fall outside of normal protocol communications, like interrupted connections, timeouts, protocol errors.

Exposes the following attributes:

flow: Flow object msg: Message describing the error timestamp: Seconds since the epoch