.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "glr_examples/applications/plot_tensor_restoring.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_glr_examples_applications_plot_tensor_restoring.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_glr_examples_applications_plot_tensor_restoring.py:


=================================================
Restoring tensor with noised AMT data
=================================================

Gives a step-by-step guide for restoring Z tensors 
using  real noised data containing missing 
and weak frequency signals. 

.. GENERATED FROM PYTHON SOURCE LINES 10-13

.. code-block:: Python

    # Author: L.Kouadio 
    # Licence: BSD-3-clause








.. GENERATED FROM PYTHON SOURCE LINES 14-34

This is an example of restoring tensors when data contains missing frequency.
The data used for the demonstration is from Huayuan data, Hunan province, China.
The survey is :term:`AMT` and data is collected alongside two long lines 
`E1` (2.5km) and `E2` (1.8km) . Unfortunately, because of strong 
interferences and the human-made made noises (agglomerations, power lines)  
and many factories in this area,  signals were strongly corrupted and greatly 
affected by strong interferences. 

* Why recovering tensors is important? 

Commonly, a missing and weak signal can just be removed from the whole data and 
keep only the valid signals from interpolation. However, in the area with 
strong interferences when data is too much noised, suppressing the weak 
signals and missing frequencies will greatly affect the quality of data 
thereby leading to a misinterpretation of the structures found in this area
after inversion. 

For demonstration, we collected 47 samples of raw data stored as an inner dataset 
and plot the raw data of impedance tensor z in TM mode to visualize the 
missing tensors. This is the code snippet: 

.. GENERATED FROM PYTHON SOURCE LINES 34-40

.. code-block:: Python

    import watex  as wx 
    from watex.methods import EMAP 
    from watex.view import plot2d 
    data = wx.fetch_data('huayuan', return_data =True, samples =47 ,
                         key ='raw', clear_cache=True) # clear watex cache data to save new EDIs
    tro = EMAP().fit(data) 







.. GENERATED FROM PYTHON SOURCE LINES 41-47

* Output the impedance tensor in TM mode (`yx`)

Here we output the modulus of the complex number of tensor Z. We can also output 
z as the complex number by setting ``out=z`` since the `kind` param is by 
default set to ``complex``. Use  ``'real'`` or ``’imag’`` for the real and imaginary 
part instead. Refer to method documentation for more details. 

.. GENERATED FROM PYTHON SOURCE LINES 47-49

.. code-block:: Python

    z_yx = tro.make2d(out= 'zyx', kind ='modulus' )  
 







.. GENERATED FROM PYTHON SOURCE LINES 50-51

* Visualize the plot using the template 2D from :func:`watex.view.plot2d`.  

.. GENERATED FROM PYTHON SOURCE LINES 51-64

.. code-block:: Python

    plot2d(z_yx, 
        y = tro.freqs_,
        to_log10= True,
        top_label='Stations', 
        plt_style ='imshow', 
        fig_size =(10, 4 ), 
        font_size =7, 
        ylabel ='Frequency[$H_z$]', 
        cb_label ='TM mode: $Z_yx$', 
        distance =50., # distance between stations
        cmap = 'terrain'
        ) 




.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_tensor_restoring_001.png
   :alt: plot tensor restoring
   :srcset: /glr_examples/applications/images/sphx_glr_plot_tensor_restoring_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <AxesSubplot:xlabel='Distance(m)', ylabel='Frequency[$H_z$]'>



.. GENERATED FROM PYTHON SOURCE LINES 65-81

As a comment: one can visualize the blank line in the data which indicates 
the missing signal at these points. If we try to remove from each missing 
signal data which corresponds to data at a certain frequency, we will lose 
a lot of useful information. Mostly the idea is to interpolate data. The 
strategy proposed by :code:`watex` for restoring tensor consists to interpolate 
in two directions (along the x-axis and y-axis ) at the same time and decide the 
best value that approximatively fits the surrounding resistivity values( because 
the demonstration refers to resistivity tensor in TM mode.). This improves a 
bit the reality of the resistivity distribution of the area. 

There is a way to do the quality control of the data before restoring using 
the quality control method :meth:`watex.methods.em.Processing.qc` by 
specifying the tolerance parameter. By default, the data is considered good 
above 50%. Note that this threshold can be severe with 30%. Indeed, the control 
for data validity should be ``soft`` when the tolerance parameter  is  close 
to '1' and ``hard`` otherwise. Here is an example to get the quality control of data. 

.. GENERATED FROM PYTHON SOURCE LINES 81-83

.. code-block:: Python

    tro.qc (tol =.4 , return_ratio = True ) # we consider good data from .60% 





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    0.94



.. GENERATED FROM PYTHON SOURCE LINES 84-93

The output shows `94%` of data goodness. This score can be improved if the 
recovers the losses signal is triggered. However, If the user approves this 
ratio, there is a possibility of outputting the valid tensors  using the 
the method :meth:`watex.methods.em.Processing.getValidTensors` and set the 
``option`` parameter to ``write`` with the same tolerance parameters as:  

.. code-block:: python 

   >>> tro.getValidTensors (tol = .4 ) # uncomment this will output new edi with valid tensors

.. GENERATED FROM PYTHON SOURCE LINES 95-97

The next step consists to recover the missing signals since we assume that 
we are not satisfied with our qc value = 94% 

.. GENERATED FROM PYTHON SOURCE LINES 97-101

.. code-block:: Python


    # * Recovering missing signals  
    Z = tro.zrestore ( ) 








.. GENERATED FROM PYTHON SOURCE LINES 102-104

Note that Z is  three dimensional array ( n_freqs, 2 , 2 ), we can 
collect the TM restoring tensors using the function :func:`watex.utils.get2dtensor` 

.. GENERATED FROM PYTHON SOURCE LINES 104-106

.. code-block:: Python

    z_yx_restored = wx.get2dtensor(Z, tensor ='z', component='yx') 








.. GENERATED FROM PYTHON SOURCE LINES 107-113

The impedance tensor z in TM mode can also be output using: 

.. code-block:: python 

   >>> tro.component ='yx' 
   >>> z_yx_restored = tro.zrestore ( tensor ='z')

.. GENERATED FROM PYTHON SOURCE LINES 115-116

* Plot the recovering tensors 

.. GENERATED FROM PYTHON SOURCE LINES 116-127

.. code-block:: Python

    wx.view.plot2d(z_yx_restored,
                   y = tro.freqs_,
                   to_log10= True,
                   top_label='Stations', 
                   plt_style ='imshow', 
                   fig_size =(10, 4 ), 
                   font_size =7, 
                   ylabel ='Frequency[$H_z$]', 
                   cb_label ='TM mode: $Z_yx$', 
                   distance =50.,cmap = 'terrain'
                   )



.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_tensor_restoring_002.png
   :alt: plot tensor restoring
   :srcset: /glr_examples/applications/images/sphx_glr_plot_tensor_restoring_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <AxesSubplot:xlabel='Distance(m)', ylabel='Frequency[$H_z$]'>



.. GENERATED FROM PYTHON SOURCE LINES 128-130

The plot below indicates the full-strength amplitudes of restored data. 
Tensors are recovered at all frequencies. 


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 3.418 seconds)


.. _sphx_glr_download_glr_examples_applications_plot_tensor_restoring.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/watex/watex/0.3.X?urlpath=lab/tree/notebooks/glr_examples/applications/plot_tensor_restoring.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_tensor_restoring.ipynb <plot_tensor_restoring.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_tensor_restoring.py <plot_tensor_restoring.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_