.. _gmr:

=========================
 Guru Meditation Reports
=========================

A Guru Meditation Report (GMR) is generated by the Designate services when
service processes receiving SIGUSR2 signal. The report is a general-purpose
debug report for developers and system admins which contains the current state
of a running Designate service process.

Structure of a GMR
==================

Package
  Shows information about the package to which this process belongs, including
  version information

Threads
  Shows stack traces and thread ids for each of the threads within this process

Green Threads
  Shows stack traces for each of the green threads within this process (green
  threads don't have thread ids)

Processes
  Shows information about this process, including pid, ppid, uid and process
  state

Configuration
  Lists all the configuration options currently accessible via the CONF object
  for the current process

Generate a GMR
==============

A GMR can be generated by sending the USR2 signal to any Designate processes.

For example, suppose ``designate-central`` has pid ``15097``, ``kill -USR2
15097`` will trigger a GMR.

If option ``logdir`` has been set in ``designate.conf``, the GMR will be saved
in the folder which ``logdir`` specified. Otherwise, the GMR will be printed to
the stderr.

Reference
=========

For more information about GMR, see `GMR wiki`_.

.. _GMR wiki: https://wiki.openstack.org/wiki/GuruMeditationReport

GMR Example
===========

::

    ========================================================================
    ====                        Guru Meditation                         ====
    ========================================================================
    ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||


    ========================================================================
    ====                            Package                             ====
    ========================================================================
    product = OpenStack Designate
    vendor = OpenStack Foundation
    version = 2015.1
    ========================================================================
    ====                            Threads                             ====
    ========================================================================
    ------                  Thread #140098874533632                   ------

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py:346 in run
        `self.wait(sleep_time)`

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py:85 in wait
        `presult = self.do_poll(seconds)`

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/epolls.py:62 in do_poll
        `return self.poll.poll(seconds)`

    ========================================================================
    ====                         Green Threads                          ====
    ========================================================================
    ------                        Green Thread                        ------

    /usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py:214 in main
        `result = function(*args, **kwargs)`

    /opt/stack/designate/designate/openstack/common/service.py:492 in run_service
        `done.wait()`

    /usr/local/lib/python2.7/dist-packages/eventlet/event.py:121 in wait
        `return hubs.get_hub().switch()`

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py:294 in switch
        `return self.greenlet.switch()`

    ------                        Green Thread                        ------

    /usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py:214 in main
        `result = function(*args, **kwargs)`

    /usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py:95 in inner_func
        `return infunc(*args, **kwargs)`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_executors/impl_eventlet.py:96 in _executor_thread
        `incoming = self.listener.poll()`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:121 in poll
        `self.conn.consume(limit=1, timeout=timeout)`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py:867 in consume
        `six.next(it)`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py:782 in iterconsume
        `yield self.ensure(_error_callback, _consume)`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py:688 in ensure
        `ret, channel = autoretry_method()`

    /usr/local/lib/python2.7/dist-packages/kombu/connection.py:436 in _ensured
        `return fun(*args, **kwargs)`

    /usr/local/lib/python2.7/dist-packages/kombu/connection.py:508 in __call__
        `return fun(*args, channel=channels[0], **kwargs), channels[0]`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py:675 in execute_method
        `method()`

    /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/impl_rabbit.py:774 in _consume
        `return self.connection.drain_events(timeout=poll_timeout)`

    /usr/local/lib/python2.7/dist-packages/kombu/connection.py:275 in drain_events
        `return self.transport.drain_events(self.connection, **kwargs)`

    /usr/local/lib/python2.7/dist-packages/kombu/transport/pyamqp.py:91 in drain_events
        `return connection.drain_events(**kwargs)`

    /usr/local/lib/python2.7/dist-packages/amqp/connection.py:302 in drain_events
        `chanmap, None, timeout=timeout,`

    /usr/local/lib/python2.7/dist-packages/amqp/connection.py:365 in _wait_multiple
        `channel, method_sig, args, content = read_timeout(timeout)`

    /usr/local/lib/python2.7/dist-packages/amqp/connection.py:336 in read_timeout
        `return self.method_reader.read_method()`

    /usr/local/lib/python2.7/dist-packages/amqp/method_framing.py:186 in read_method
        `self._next_method()`

    /usr/local/lib/python2.7/dist-packages/amqp/method_framing.py:107 in _next_method
        `frame_type, channel, payload = read_frame()`

    /usr/local/lib/python2.7/dist-packages/amqp/transport.py:154 in read_frame
        `frame_header = read(7, True)`

    /usr/local/lib/python2.7/dist-packages/amqp/transport.py:277 in _read
        `s = recv(n - len(rbuf))`

    /usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py:326 in recv
        `timeout_exc=socket.timeout("timed out"))`

    /usr/local/lib/python2.7/dist-packages/eventlet/greenio/base.py:201 in _trampoline
        `mark_as_closed=self._mark_as_closed)`

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/__init__.py:162 in trampoline
        `return hub.switch()`

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py:294 in switch
        `return self.greenlet.switch()`

    ------                        Green Thread                        ------

    /usr/local/bin/designate-central:10 in <module>
        `sys.exit(main())`

    /opt/stack/designate/designate/cmd/central.py:37 in main
        `service.wait()`

    /opt/stack/designate/designate/service.py:356 in wait
        `_launcher.wait()`

    /opt/stack/designate/designate/openstack/common/service.py:187 in wait
        `status, signo = self._wait_for_exit_or_signal(ready_callback)`

    /opt/stack/designate/designate/openstack/common/service.py:170 in _wait_for_exit_or_signal
        `super(ServiceLauncher, self).wait()`

    /opt/stack/designate/designate/openstack/common/service.py:133 in wait
        `self.services.wait()`

    /opt/stack/designate/designate/openstack/common/service.py:473 in wait
        `self.tg.wait()`

    /opt/stack/designate/designate/openstack/common/threadgroup.py:145 in wait
        `x.wait()`

    /opt/stack/designate/designate/openstack/common/threadgroup.py:47 in wait
        `return self.thread.wait()`

    /usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py:175 in wait
        `return self._exit_event.wait()`

    /usr/local/lib/python2.7/dist-packages/eventlet/event.py:121 in wait
        `return hubs.get_hub().switch()`

    /usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py:294 in switch
        `return self.greenlet.switch()`

    ------                        Green Thread                        ------

    No Traceback!

    ========================================================================
    ====                           Processes                            ====
    ========================================================================
    Process 15097 (under 7312) [ run by: stanzgy (1000), state: running ]

    ========================================================================
    ====                         Configuration                          ====
    ========================================================================

    backend:agent:bind9:
      query-destination = 127.0.0.1
      rndc-config-file = None
      rndc-host = 127.0.0.1
      rndc-key-file = None
      rndc-port = 953
      zone-file-path = /opt/stack/data/designate/zones

    backend:bind9:
      masters =
        127.0.0.1:5354
      rndc-config-file = None
      rndc-host = 127.0.0.1
      rndc-key-file = None
      rndc-port = 953
      server_ids =

    backend:fake:
      masters =
        127.0.0.1:5354
      server_ids =

    backend:powerdns:
      backend = sqlalchemy
      connection = ***
      connection_debug = 0
      connection_trace = False
      db_inc_retry_interval = True
      db_max_retries = 20
      db_max_retry_interval = 10
      db_retry_interval = 1
      idle_timeout = 3600
      masters =
        10.180.64.117:5354
      max_overflow = None
      max_pool_size = None
      max_retries = 10
      min_pool_size = 1
      mysql_sql_mode = TRADITIONAL
      pool_timeout = None
      retry_interval = 10
      server_ids =
        f26e0b32-736f-4f0a-831b-039a415c481e
      slave_connection = ***
      sqlite_db = oslo.sqlite
      sqlite_synchronous = True
      use_db_reconnect = False

    backend:powerdns:f26e0b32-736f-4f0a-831b-039a415c481e:
      backend = None
      connection = ***
      connection_debug = None
      connection_trace = None
      db_inc_retry_interval = None
      db_max_retries = None
      db_max_retry_interval = None
      db_retry_interval = None
      host = 10.180.64.117
      idle_timeout = None
      masters = None
      max_overflow = None
      max_pool_size = None
      max_retries = None
      min_pool_size = None
      mysql_sql_mode = None
      pool_timeout = None
      port = 53
      retry_interval = None
      slave_connection = ***
      sqlite_db = None
      sqlite_synchronous = None
      tsig-key = None
      use_db_reconnect = None

    default:
      allowed_remote_exmods =
      backdoor_port = None
      backlog = 4096
      central-topic = central
      config-dir = None
      config-file =
        /etc/designate/designate.conf
      control_exchange = designate
      debug = True
      default-soa-expire = 86400
      default-soa-minimum = 3600
      default-soa-refresh-min = 3500
      default-soa-refresh-max = 3600
      default-soa-retry = 600
      default-ttl = 3600
      default_log_levels =
        amqp=WARN
        amqplib=WARN
        boto=WARN
        eventlet.wsgi.server=WARN
        keystone=INFO
        keystonemiddleware.auth_token=INFO
        oslo.messaging=WARN
        sqlalchemy=WARN
        stevedore=WARN
        suds=INFO
      fatal_deprecations = False
      host = cns-dev2
      instance_format = [instance: %(uuid)s]
      instance_uuid_format = [instance: %(uuid)s]
      log-config-append = None
      log-date-format = %Y-%m-%d %H:%M:%S
      log-dir = /opt/stack/logs/designate
      log-file = None
      log-format = None
      logging_context_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [[01;36m%(request_id)s [00;36m%(user)s %(tenant)s%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m
      logging_debug_format_suffix = [00;33mfrom (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d[00m
      logging_default_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [[00;36m-%(color)s] [01;35m%(instance)s%(color)s%(message)s[00m
      logging_exception_prefix = %(color)s%(asctime)s.%(msecs)03d TRACE %(name)s [01;35m%(instance)s[00m
      mdns-topic = mdns
      network_api = neutron
      notification_driver =
      notification_topics =
        notifications
      policy_default_rule = default
      policy_dirs =
        policy.d
      policy_file = /etc/designate/policy.yaml
      pool-manager-topic = pool_manager
      publish_errors = False
      pybasedir = /opt/stack/designate
      quota-domain-records = 500
      quota-domain-recordsets = 500
      quota-domains = 10
      quota-driver = storage
      quota-recordset-records = 20
      root-helper = sudo designate-rootwrap /etc/designate/rootwrap.conf
      rpc_backend = rabbit
      rpc_thread_pool_size = 64
      state-path = /opt/stack/data/designate
      syslog-log-facility = LOG_USER
      tcp_keepidle = 600
      transport_url = None
      use-syslog = False
      use-syslog-rfc-format = False
      use_stderr = True
      verbose = True

    network_api:neutron:
      admin_password = ***
      admin_tenant_name = None
      admin_username = None
      auth_strategy = keystone
      auth_url = None
      ca_certificates_file = None
      endpoint_type = publicURL
      endpoints = None
      insecure = False
      timeout = 30

    oslo_concurrency:
      disable_process_locking = False
      lock_path = None

    oslo_messaging_rabbit:
      amqp_auto_delete = False
      amqp_durable_queues = False
      fake_rabbit = False
      kombu_reconnect_delay = 1.0
      kombu_ssl_ca_certs =
      kombu_ssl_certfile =
      kombu_ssl_keyfile =
      kombu_ssl_version =
      rabbit_ha_queues = False
      rabbit_host = localhost
      rabbit_hosts =
        127.0.0.1
      rabbit_login_method = AMQPLAIN
      rabbit_max_retries = 0
      rabbit_password = ***
      rabbit_port = 5672
      rabbit_retry_backoff = 2
      rabbit_retry_interval = 1
      rabbit_use_ssl = False
      rabbit_userid = stackrabbit
      rabbit_virtual_host = /
      rpc_conn_pool_size = 30

    proxy:
      http_proxy = None
      https_proxy = None
      no_proxy =

    service:central:
      default_pool_id = 794ccc2c-d751-44fe-b57f-8894c9f5c842
      enabled-notification-handlers =
      managed_resource_email = hostmaster@example.com
      managed_resource_tenant_id = None
      max_domain_name_len = 255
      max_recordset_name_len = 255
      min_ttl = None
      storage-driver = sqlalchemy
      workers = None

    service:pool_manager:
      backends =
        powerdns
      cache-driver = sqlalchemy
      enable-recovery-timer = True
      enable-sync-timer = True
      periodic-recovery-interval = 120
      periodic-sync-interval = 300
      periodic-sync-seconds = None
      poll-delay = 1
      poll-max-retries = 3
      poll-retry-interval = 2
      poll-timeout = 30
      pool-id = 794ccc2c-d751-44fe-b57f-8894c9f5c842
      threshold-percentage = 100
      workers = None

    ssl:
      ca_file = None
      cert_file = None
      key_file = None

    storage:sqlalchemy:
      backend = sqlalchemy
      connection = ***
      connection_debug = 0
      connection_trace = False
      db_inc_retry_interval = True
      db_max_retries = 20
      db_max_retry_interval = 10
      db_retry_interval = 1
      idle_timeout = 3600
      max_overflow = None
      max_pool_size = None
      max_retries = 10
      min_pool_size = 1
      mysql_sql_mode = TRADITIONAL
      pool_timeout = None
      retry_interval = 10
      slave_connection = ***
      sqlite_db = oslo.sqlite
      sqlite_synchronous = True
      use_db_reconnect = False
