
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/model_selection/plot_successive_halving_iterations.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_model_selection_plot_successive_halving_iterations.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_model_selection_plot_successive_halving_iterations.py:


Successive Halving Iterations
=============================

This example illustrates how a successive halving search
(:class:`~sklearn.model_selection.HalvingGridSearchCV` and
:class:`~sklearn.model_selection.HalvingRandomSearchCV`)
iteratively chooses the best parameter combination out of
multiple candidates.

.. GENERATED FROM PYTHON SOURCE LINES 12-24

.. code-block:: default


    import pandas as pd
    from sklearn import datasets
    import matplotlib.pyplot as plt
    from scipy.stats import randint
    import numpy as np

    from sklearn.experimental import enable_halving_search_cv  # noqa
    from sklearn.model_selection import HalvingRandomSearchCV
    from sklearn.ensemble import RandomForestClassifier









.. GENERATED FROM PYTHON SOURCE LINES 25-27

We first define the parameter space and train a
:class:`~sklearn.model_selection.HalvingRandomSearchCV` instance.

.. GENERATED FROM PYTHON SOURCE LINES 27-47

.. code-block:: default


    rng = np.random.RandomState(0)

    X, y = datasets.make_classification(n_samples=400, n_features=12, random_state=rng)

    clf = RandomForestClassifier(n_estimators=20, random_state=rng)

    param_dist = {
        "max_depth": [3, None],
        "max_features": randint(1, 6),
        "min_samples_split": randint(2, 11),
        "bootstrap": [True, False],
        "criterion": ["gini", "entropy"],
    }

    rsh = HalvingRandomSearchCV(
        estimator=clf, param_distributions=param_dist, factor=2, random_state=rng
    )
    rsh.fit(X, y)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 {color: black;background-color: white;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 pre{padding: 0;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-toggleable {background-color: white;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-estimator:hover {background-color: #d4ebff;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-item {z-index: 1;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel-item {display: flex;flex-direction: column;position: relative;background-color: white;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-parallel-item:only-child::after {width: 0;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;position: relative;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-label label {font-family: monospace;font-weight: bold;background-color: white;display: inline-block;line-height: 1.2em;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-label-container {position: relative;z-index: 2;text-align: center;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605 div.sk-text-repr-fallback {display: none;}</style><div id="sk-6c4b1a41-e08e-45f9-9c00-7ab71b472605" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>HalvingRandomSearchCV(estimator=RandomForestClassifier(n_estimators=20,
                                                           random_state=RandomState(MT19937) at 0x7F38D5D25440),
                          factor=2,
                          param_distributions={&#x27;bootstrap&#x27;: [True, False],
                                               &#x27;criterion&#x27;: [&#x27;gini&#x27;, &#x27;entropy&#x27;],
                                               &#x27;max_depth&#x27;: [3, None],
                                               &#x27;max_features&#x27;: &lt;scipy.stats._distn_infrastructure.rv_frozen object at 0x7f38b7d35a60&gt;,
                                               &#x27;min_samples_split&#x27;: &lt;scipy.stats._distn_infrastructure.rv_frozen object at 0x7f38d51d2460&gt;},
                          random_state=RandomState(MT19937) at 0x7F38D5D25440)</pre><b>Please rerun this cell to show the HTML repr or trust the notebook.</b></div><div class="sk-container" hidden><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="6c5af482-cb7b-4767-9a95-002280a07a89" type="checkbox" ><label for="6c5af482-cb7b-4767-9a95-002280a07a89" class="sk-toggleable__label sk-toggleable__label-arrow">HalvingRandomSearchCV</label><div class="sk-toggleable__content"><pre>HalvingRandomSearchCV(estimator=RandomForestClassifier(n_estimators=20,
                                                           random_state=RandomState(MT19937) at 0x7F38D5D25440),
                          factor=2,
                          param_distributions={&#x27;bootstrap&#x27;: [True, False],
                                               &#x27;criterion&#x27;: [&#x27;gini&#x27;, &#x27;entropy&#x27;],
                                               &#x27;max_depth&#x27;: [3, None],
                                               &#x27;max_features&#x27;: &lt;scipy.stats._distn_infrastructure.rv_frozen object at 0x7f38b7d35a60&gt;,
                                               &#x27;min_samples_split&#x27;: &lt;scipy.stats._distn_infrastructure.rv_frozen object at 0x7f38d51d2460&gt;},
                          random_state=RandomState(MT19937) at 0x7F38D5D25440)</pre></div></div></div><div class="sk-parallel"><div class="sk-parallel-item"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="cc521209-7d20-48eb-8a0d-462ad005a43c" type="checkbox" ><label for="cc521209-7d20-48eb-8a0d-462ad005a43c" class="sk-toggleable__label sk-toggleable__label-arrow">RandomForestClassifier</label><div class="sk-toggleable__content"><pre>RandomForestClassifier(n_estimators=20,
                           random_state=RandomState(MT19937) at 0x7F38D5D25440)</pre></div></div></div></div></div></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 48-50

We can now use the `cv_results_` attribute of the search estimator to inspect
and plot the evolution of the search.

.. GENERATED FROM PYTHON SOURCE LINES 50-72

.. code-block:: default


    results = pd.DataFrame(rsh.cv_results_)
    results["params_str"] = results.params.apply(str)
    results.drop_duplicates(subset=("params_str", "iter"), inplace=True)
    mean_scores = results.pivot(
        index="iter", columns="params_str", values="mean_test_score"
    )
    ax = mean_scores.plot(legend=False, alpha=0.6)

    labels = [
        f"iter={i}\nn_samples={rsh.n_resources_[i]}\nn_candidates={rsh.n_candidates_[i]}"
        for i in range(rsh.n_iterations_)
    ]

    ax.set_xticks(range(rsh.n_iterations_))
    ax.set_xticklabels(labels, rotation=45, multialignment="left")
    ax.set_title("Scores of candidates over iterations")
    ax.set_ylabel("mean test score", fontsize=15)
    ax.set_xlabel("iterations", fontsize=15)
    plt.tight_layout()
    plt.show()




.. image-sg:: /auto_examples/model_selection/images/sphx_glr_plot_successive_halving_iterations_001.png
   :alt: Scores of candidates over iterations
   :srcset: /auto_examples/model_selection/images/sphx_glr_plot_successive_halving_iterations_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 73-87

Number of candidates and amount of resource at each iteration
-------------------------------------------------------------

At the first iteration, a small amount of resources is used. The resource
here is the number of samples that the estimators are trained on. All
candidates are evaluated.

At the second iteration, only the best half of the candidates is evaluated.
The number of allocated resources is doubled: candidates are evaluated on
twice as many samples.

This process is repeated until the last iteration, where only 2 candidates
are left. The best candidate is the candidate that has the best score at the
last iteration.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  4.376 seconds)


.. _sphx_glr_download_auto_examples_model_selection_plot_successive_halving_iterations.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_successive_halving_iterations.py <plot_successive_halving_iterations.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_successive_halving_iterations.ipynb <plot_successive_halving_iterations.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
