
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/text/plot_document_classification_20newsgroups.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_text_plot_document_classification_20newsgroups.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_text_plot_document_classification_20newsgroups.py:


======================================================
Classification of text documents using sparse features
======================================================

This is an example showing how scikit-learn can be used to classify documents
by topics using a bag-of-words approach. This example uses a scipy.sparse
matrix to store the features and demonstrates various classifiers that can
efficiently handle sparse matrices.

The dataset used in this example is the 20 newsgroups dataset. It will be
automatically downloaded, then cached.

.. GENERATED FROM PYTHON SOURCE LINES 15-23

.. code-block:: default


    # Author: Peter Prettenhofer <peter.prettenhofer@gmail.com>
    #         Olivier Grisel <olivier.grisel@ensta.org>
    #         Mathieu Blondel <mathieu@mblondel.org>
    #         Lars Buitinck
    # License: BSD 3 clause









.. GENERATED FROM PYTHON SOURCE LINES 24-26

Configuration options for the analysis
--------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 26-38

.. code-block:: default


    # If True, we use `HashingVectorizer`, otherwise we use a `TfidfVectorizer`
    USE_HASHING = False

    # Number of features used by `HashingVectorizer`
    N_FEATURES = 2**16

    # Optional feature selection: either False, or an integer: the number of
    # features to select
    SELECT_CHI2 = False









.. GENERATED FROM PYTHON SOURCE LINES 39-44

Load data from the training set
------------------------------------
Let's load data from the newsgroups dataset which comprises around 18000
newsgroups posts on 20 topics split in two subsets: one for training (or
development) and the other one for testing (or for performance evaluation).

.. GENERATED FROM PYTHON SOURCE LINES 44-79

.. code-block:: default

    from sklearn.datasets import fetch_20newsgroups

    categories = [
        "alt.atheism",
        "talk.religion.misc",
        "comp.graphics",
        "sci.space",
    ]

    data_train = fetch_20newsgroups(
        subset="train", categories=categories, shuffle=True, random_state=42
    )

    data_test = fetch_20newsgroups(
        subset="test", categories=categories, shuffle=True, random_state=42
    )
    print("data loaded")

    # order of labels in `target_names` can be different from `categories`
    target_names = data_train.target_names


    def size_mb(docs):
        return sum(len(s.encode("utf-8")) for s in docs) / 1e6


    data_train_size_mb = size_mb(data_train.data)
    data_test_size_mb = size_mb(data_test.data)

    print(
        "%d documents - %0.3fMB (training set)" % (len(data_train.data), data_train_size_mb)
    )
    print("%d documents - %0.3fMB (test set)" % (len(data_test.data), data_test_size_mb))
    print("%d categories" % len(target_names))



.. rst-class:: sphx-glr-script-out

.. code-block:: pytb

    Traceback (most recent call last):
      File "/build/scikit-learn-TElydD/scikit-learn-1.1.1/examples/text/plot_document_classification_20newsgroups.py", line 53, in <module>
        data_train = fetch_20newsgroups(
      File "/build/scikit-learn-TElydD/scikit-learn-1.1.1/.pybuild/cpython3_3.10/build/sklearn/datasets/_twenty_newsgroups.py", line 269, in fetch_20newsgroups
        cache = _download_20newsgroups(
      File "/build/scikit-learn-TElydD/scikit-learn-1.1.1/.pybuild/cpython3_3.10/build/sklearn/datasets/_twenty_newsgroups.py", line 74, in _download_20newsgroups
        archive_path = _fetch_remote(ARCHIVE, dirname=target_dir)
      File "/build/scikit-learn-TElydD/scikit-learn-1.1.1/.pybuild/cpython3_3.10/build/sklearn/datasets/_base.py", line 1510, in _fetch_remote
        raise IOError('Debian Policy Section 4.9 prohibits network access during build')
    OSError: Debian Policy Section 4.9 prohibits network access during build




.. GENERATED FROM PYTHON SOURCE LINES 80-84

Vectorize the training and test data
-------------------------------------

split a training set and a test set

.. GENERATED FROM PYTHON SOURCE LINES 84-86

.. code-block:: default

    y_train, y_test = data_train.target, data_test.target


.. GENERATED FROM PYTHON SOURCE LINES 87-88

Extracting features from the training data using a sparse vectorizer

.. GENERATED FROM PYTHON SOURCE LINES 88-107

.. code-block:: default

    from time import time

    from sklearn.feature_extraction.text import TfidfVectorizer
    from sklearn.feature_extraction.text import HashingVectorizer

    t0 = time()

    if USE_HASHING:
        vectorizer = HashingVectorizer(
            stop_words="english", alternate_sign=False, n_features=N_FEATURES
        )
        X_train = vectorizer.transform(data_train.data)
    else:
        vectorizer = TfidfVectorizer(sublinear_tf=True, max_df=0.5, stop_words="english")
        X_train = vectorizer.fit_transform(data_train.data)
    duration = time() - t0
    print("done in %fs at %0.3fMB/s" % (duration, data_train_size_mb / duration))
    print("n_samples: %d, n_features: %d" % X_train.shape)


.. GENERATED FROM PYTHON SOURCE LINES 108-109

Extracting features from the test data using the same vectorizer

.. GENERATED FROM PYTHON SOURCE LINES 109-115

.. code-block:: default

    t0 = time()
    X_test = vectorizer.transform(data_test.data)
    duration = time() - t0
    print("done in %fs at %0.3fMB/s" % (duration, data_test_size_mb / duration))
    print("n_samples: %d, n_features: %d" % X_test.shape)


.. GENERATED FROM PYTHON SOURCE LINES 116-117

mapping from integer feature name to original token string

.. GENERATED FROM PYTHON SOURCE LINES 117-122

.. code-block:: default

    if USE_HASHING:
        feature_names = None
    else:
        feature_names = vectorizer.get_feature_names_out()


.. GENERATED FROM PYTHON SOURCE LINES 123-124

Keeping only the best features

.. GENERATED FROM PYTHON SOURCE LINES 124-139

.. code-block:: default

    from sklearn.feature_selection import SelectKBest, chi2

    if SELECT_CHI2:
        print("Extracting %d best features by a chi-squared test" % SELECT_CHI2)
        t0 = time()
        ch2 = SelectKBest(chi2, k=SELECT_CHI2)
        X_train = ch2.fit_transform(X_train, y_train)
        X_test = ch2.transform(X_test)
        if feature_names is not None:
            # keep selected feature names
            feature_names = feature_names[ch2.get_support()]
        print("done in %fs" % (time() - t0))
        print()



.. GENERATED FROM PYTHON SOURCE LINES 140-144

Benchmark classifiers
------------------------------------

First we define small benchmarking utilities

.. GENERATED FROM PYTHON SOURCE LINES 144-193

.. code-block:: default

    import numpy as np
    from sklearn import metrics
    from sklearn.utils.extmath import density


    def trim(s):
        """Trim string to fit on terminal (assuming 80-column display)"""
        return s if len(s) <= 80 else s[:77] + "..."


    def benchmark(clf):
        print("_" * 80)
        print("Training: ")
        print(clf)
        t0 = time()
        clf.fit(X_train, y_train)
        train_time = time() - t0
        print("train time: %0.3fs" % train_time)

        t0 = time()
        pred = clf.predict(X_test)
        test_time = time() - t0
        print("test time:  %0.3fs" % test_time)

        score = metrics.accuracy_score(y_test, pred)
        print("accuracy:   %0.3f" % score)

        if hasattr(clf, "coef_"):
            print("dimensionality: %d" % clf.coef_.shape[1])
            print("density: %f" % density(clf.coef_))

            if feature_names is not None:
                print("top 10 keywords per class:")
                for i, label in enumerate(target_names):
                    top10 = np.argsort(clf.coef_[i])[-10:]
                    print(trim("%s: %s" % (label, " ".join(feature_names[top10]))))
            print()

        print("classification report:")
        print(metrics.classification_report(y_test, pred, target_names=target_names))

        print("confusion matrix:")
        print(metrics.confusion_matrix(y_test, pred))

        print()
        clf_descr = str(clf).split("(")[0]
        return clf_descr, score, train_time, test_time



.. GENERATED FROM PYTHON SOURCE LINES 194-196

We now train and test the datasets with 15 different classification
models and get performance results for each model.

.. GENERATED FROM PYTHON SOURCE LINES 196-268

.. code-block:: default

    from sklearn.feature_selection import SelectFromModel
    from sklearn.linear_model import RidgeClassifier
    from sklearn.pipeline import Pipeline
    from sklearn.svm import LinearSVC
    from sklearn.linear_model import SGDClassifier
    from sklearn.linear_model import Perceptron
    from sklearn.linear_model import PassiveAggressiveClassifier
    from sklearn.naive_bayes import BernoulliNB, ComplementNB, MultinomialNB
    from sklearn.neighbors import KNeighborsClassifier
    from sklearn.neighbors import NearestCentroid
    from sklearn.ensemble import RandomForestClassifier


    results = []
    for clf, name in (
        (RidgeClassifier(tol=1e-2, solver="sag"), "Ridge Classifier"),
        (Perceptron(max_iter=50), "Perceptron"),
        (PassiveAggressiveClassifier(max_iter=50), "Passive-Aggressive"),
        (KNeighborsClassifier(n_neighbors=10), "kNN"),
        (RandomForestClassifier(), "Random forest"),
    ):
        print("=" * 80)
        print(name)
        results.append(benchmark(clf))

    for penalty in ["l2", "l1"]:
        print("=" * 80)
        print("%s penalty" % penalty.upper())
        # Train Liblinear model
        results.append(benchmark(LinearSVC(penalty=penalty, dual=False, tol=1e-3)))

        # Train SGD model
        results.append(benchmark(SGDClassifier(alpha=0.0001, max_iter=50, penalty=penalty)))

    # Train SGD with Elastic Net penalty
    print("=" * 80)
    print("Elastic-Net penalty")
    results.append(
        benchmark(SGDClassifier(alpha=0.0001, max_iter=50, penalty="elasticnet"))
    )

    # Train NearestCentroid without threshold
    print("=" * 80)
    print("NearestCentroid (aka Rocchio classifier)")
    results.append(benchmark(NearestCentroid()))

    # Train sparse Naive Bayes classifiers
    print("=" * 80)
    print("Naive Bayes")
    results.append(benchmark(MultinomialNB(alpha=0.01)))
    results.append(benchmark(BernoulliNB(alpha=0.01)))
    results.append(benchmark(ComplementNB(alpha=0.1)))

    print("=" * 80)
    print("LinearSVC with L1-based feature selection")
    # The smaller C, the stronger the regularization.
    # The more regularization, the more sparsity.
    results.append(
        benchmark(
            Pipeline(
                [
                    (
                        "feature_selection",
                        SelectFromModel(LinearSVC(penalty="l1", dual=False, tol=1e-3)),
                    ),
                    ("classification", LinearSVC(penalty="l2")),
                ]
            )
        )
    )



.. GENERATED FROM PYTHON SOURCE LINES 269-273

Add plots
------------------------------------
The bar plot indicates the accuracy, training time (normalized) and test time
(normalized) of each classifier.

.. GENERATED FROM PYTHON SOURCE LINES 273-298

.. code-block:: default

    import matplotlib.pyplot as plt

    indices = np.arange(len(results))

    results = [[x[i] for x in results] for i in range(4)]

    clf_names, score, training_time, test_time = results
    training_time = np.array(training_time) / np.max(training_time)
    test_time = np.array(test_time) / np.max(test_time)

    plt.figure(figsize=(12, 8))
    plt.title("Score")
    plt.barh(indices, score, 0.2, label="score", color="navy")
    plt.barh(indices + 0.3, training_time, 0.2, label="training time", color="c")
    plt.barh(indices + 0.6, test_time, 0.2, label="test time", color="darkorange")
    plt.yticks(())
    plt.legend(loc="best")
    plt.subplots_adjust(left=0.25)
    plt.subplots_adjust(top=0.95)
    plt.subplots_adjust(bottom=0.05)

    for i, c in zip(indices, clf_names):
        plt.text(-0.3, i, c)

    plt.show()


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.003 seconds)


.. _sphx_glr_download_auto_examples_text_plot_document_classification_20newsgroups.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_document_classification_20newsgroups.py <plot_document_classification_20newsgroups.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_document_classification_20newsgroups.ipynb <plot_document_classification_20newsgroups.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
