generic_extractor.twitter

generic_extractor.twitter#

Module Contents#

class generic_extractor.twitter.Twitter#

Bases: generic_extractor.dropin.GenericDropin

Base class for dropins for the generic extractor.

In many instances, an extractor will exist in ytdlp, but it will only process videos. Dropins can be created and used to make use of the already-written private code of a specific extractor from ytdlp.

The dropin should be able to handle the following methods:

  • get_post_data: This method should be able to extract the post data from the url and return it as a dict.

  • create_metadata: This method should be able to create a Metadata object from a post dict.

Optional methods include:

  • skip_ytdlp_download: If you want to skip the ytdlp ‘download’ method all together, and do your own, then return True for this method.

    This is useful in cases where ytdlp might not work properly for all of your posts

  • keys_to_clean: for the generic ‘video_data’ created by ytdlp (for video URLs), any additional fields you would like to clean out of the data before storing in metadata

choose_variant(variants)#
extract_post(url: str, ie_instance: generic_extractor.dropin.InfoExtractor)#

This method should return the post data from the url.

keys_to_clean(video_data, info_extractor)#

This method should return a list of strings (keys) to clean from the video_data dict.

E.g. [“uploader”, “uploader_id”, “tiktok_specific_field”]

create_metadata(tweet: dict, ie_instance: generic_extractor.dropin.InfoExtractor, archiver: auto_archiver.core.extractor.Extractor, url: str) → auto_archiver.core.metadata.Metadata#

This method should create a Metadata object from the post data.

Contents