utils

utils#

Auto Archiver Utilities.

Submodules#

Package Contents#

utils.mkdir_if_not_exists(folder)#
utils.expand_url(url)#
utils.getattr_or(o: object, prop: str, default=None)#
class utils.DateTimeEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)#

Bases: json.JSONEncoder

Extensible JSON <https://json.org> encoder for Python data structures.

Supports the following objects and types by default:

Python

JSON

dict

object

list, tuple

array

str

string

int, float

number

True

true

False

false

None

null

To extend this to recognize other objects, subclass and implement a .default() method with another method that returns a serializable object for o if possible, otherwise it should call the superclass implementation (to raise TypeError).

default(o)#

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)
utils.dump_payload(p)#
utils.update_nested_dict(dictionary, update_dict)#
utils.random_str(length: int = 32) → str#
utils.calculate_file_hash(filename: str, hash_algo=hashlib.sha256, chunksize: int = 16000000) → str#
utils.get_datetime_from_str(dt_str: str, fmt: str | None = None, dayfirst=True) → datetime.datetime | None#

parse a datetime string with option of passing a specific format

Parameters:

  • dt_str – the datetime string to parse

  • fmt – the python date format of the datetime string, if None, dateutil.parser.parse is used

  • dayfirst – Use this to signify between date formats which put the day first, vs the month first: e.g. DD/MM/YYYY vs MM/DD/YYYY

utils.get_timestamp(ts, utc=True, iso=True, dayfirst=True) → str | datetime.datetime | None#

Consistent parsing of timestamps. :param If utc=True: :param the timezone is set to UTC: :param : :param if iso=True: :param the output is an iso string: :param Use dayfirst to signify between date formats which put the date vs month first: :param e.g. DD/MM/YYYY vs MM/DD/YYYY:

utils.get_current_timestamp() → str#
class utils.Webdriver(width: int, height: int, timeout_seconds: int, facebook_accept_cookies: bool = False, http_proxy: str = '', print_options: dict = {}, auth: dict = {})#
width#
height#
timeout_seconds#
auth#
facebook_accept_cookies = False#
http_proxy = ''#
print_options#
Contents