PyFunceble.downloader package

Submodules

PyFunceble.downloader.base module

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the base of all our downloader.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.downloader.base.DownloaderBase[source]

Bases: object

Provides the base of all downloader classes.

The interface is actually simple, but the part which may be hard to understand is the “downtime” part. What we do, is that we save the download time inside a JSON file, so this class provides the base around the download mechanism but also the generation or update of that JSON file.

DOWNLOAD_FREQUENCY: int = 1

The download frequency (in day).

Example:: if 1 is given, it’s once every 24 hours.

Warning

A frequency of 0 or a negative number will force the download every hour.

DOWNTIME_INDEX: str | None = None: Used to set/track the download time of the current file.

all_downtimes: dict | None = {}: Stores the download time of all files (self managed).

property authorized: bool: Provides the authorization to start the download.

property destination: str | None: Provides the current state of the _destination attribute.

dict_helper: DictHelper = <PyFunceble.helpers.dict.DictHelper object>

property download_link: str | None: Provides the current state of the _download_link attribute.

get_all_downtimes() → dict[source]: Provides the downloadtime of all files.

get_current_downtime()[source]: Provides the download times of the current DOWN_TIME_INDEX.

is_downtime_set() → bool[source]: Checks if the download time of the current object exists.

is_last_download_expired() → bool[source]: Checks if the last downloaded file is expired (if exists).

save_all_downtimes() → None[source]: Saves the current state of the all downtimes.

set_current_downtime() → DownloaderBase[source]: Sets the current datetime into our registry.

set_destination(value: str) → DownloaderBase[source]

Sets the destination.

Parameters:: value – The value to set.

set_download_link(value: str) → DownloaderBase[source]

Sets the link to download.

Parameters:: value – The value to set.

start() → None[source]: Starts the download process.

PyFunceble.downloader.exceptions module

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the exceptions related to the downloader.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

exception PyFunceble.downloader.exceptions.NoDownloadDestinationGiven[source]

Bases: PyFuncebleDownloaderException

Describes the fact that the download destination was not declared.

exception PyFunceble.downloader.exceptions.NoDownloadLinkGiven[source]

Bases: PyFuncebleDownloaderException

Describes the fact that no download link was declared.

exception PyFunceble.downloader.exceptions.PyFuncebleDownloaderException[source]

Bases: PyFuncebleException

Describes the downloader (related) exceptions.

PyFunceble.downloader.iana module

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the downloader of the latest iana database file.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.downloader.iana.IANADownloader[source]

Bases: DownloaderBase

Provides the downloader of our iana file.

DOWNLOAD_FREQUENCY: int = 1

The download frequency (in day).

Example:: if 1 is given, it’s once every 24 hours.

Warning

A frequency of 0 or a negative number will force the download every hour.

DOWNTIME_INDEX: str = 'iana': Used to set/track the download time of the current file.

property authorized: bool: Provides the authorization to start the download.

PyFunceble.downloader.ipv4_reputation module

PyFunceble.downloader.public_suffix module

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the downloader of the latest iana database file.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.downloader.public_suffix.PublicSuffixDownloader[source]

Bases: DownloaderBase

Provides the downloader of our public suffix file.

DOWNLOAD_FREQUENCY: int = 1

The download frequency (in day).

Example:: if 1 is given, it’s once every 24 hours.

Warning

A frequency of 0 or a negative number will force the download every hour.

DOWNTIME_INDEX: str = 'psl': Used to set/track the download time of the current file.

property authorized: bool: Provides the authorization to start the download.

PyFunceble.downloader.user_agents module

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the downloader of the latest user agents database file.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.downloader.user_agents.UserAgentsDownloader[source]

Bases: DownloaderBase

Provides the downloader of our user agent file.

DOWNLOAD_FREQUENCY: int = 1

The download frequency (in day).

Example:: if 1 is given, it’s once every 24 hours.

Warning

A frequency of 0 or a negative number will force the download every hour.

DOWNTIME_INDEX: str = 'user_agents': Used to set/track the download time of the current file.

property authorized: bool: Provides the authorization to start the download.

Module contents

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides all downloaders.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.