.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "glr_examples/applications/plot_dc_em_parameters_computing.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_glr_examples_applications_plot_dc_em_parameters_computing.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_glr_examples_applications_plot_dc_em_parameters_computing.py:


==========================================
EM, DC, and Hydro parameters computing 
==========================================

Real-world examples for showing the computation of EM tensor, 
DC parameters and how to implement the mixture learning strategy (MXS) from 
the naive aquifer group (NGA) for the permeability coefficient k prediction. 

.. GENERATED FROM PYTHON SOURCE LINES 10-13

.. code-block:: Python

    # Author: L.Kouadio 
    # Licence: BSD-3-clause 








.. GENERATED FROM PYTHON SOURCE LINES 14-15

Import required modules 

.. GENERATED FROM PYTHON SOURCE LINES 15-40

.. code-block:: Python

    import numpy as np 
    import matplotlib.pyplot as plt
    plt_style ='classic'

    # Load real data collected during the drinking water supply campaign 
    # occured in 2012-2014 in Cote d'Ivoire. Read more in the dataset 
    # module ( :mod:`~watex.datasets`)
    from watex.datasets import ( 
        load_edis, 
        load_gbalo, 
        load_semien, 
        load_tankesse,
        load_boundiali
        ) 
    # Real logging data collected in Hongliu coal mine, in China, Hunan province
    from watex.datasets import load_hlogs
    from watex.methods import( 
        DCProfiling, 
        DCSounding, 
        ResistivityProfiling,
        VerticalSounding,EMAP , 
        Logging,
        MXS 
        )
 







.. GENERATED FROM PYTHON SOURCE LINES 41-65

EM :mod:`~watex.methods.em`
============================
The EM module is related to a few meter exploration in the case of groundwater 
exploration. The module provides some basic EMAP steps for EMAP data filtering
and removing noises. Commonly the method mostly used in groundwater 
exploration is the audio-magnetotelluric because of the shortest frequency 
and rapid executions. Furthermore, we can also list some other advantages 
such as:  

* is useful for imaging both deep geologic structure and near-surface geology and can provide significant details. 
* includes a backpack portable system that allows for use in difficult terrain. 
* the technique requires no high-voltage electrodes, and logistics are 
  relatively easy to support in the field. Stations can be acquired almost 
  anywhere and can be placed any distance apart. This allows for large-scale 
  regional reconnaissance exploration or detailed surveys of local geology 
  and has no environmental impact 

:note: For deep implementation or exploring a large scale of EM/AMT data  
    EMAP, it is recommended to use the package `pycsamt <https://github.com/WEgeophysics/watex/>`_. 
    Create EM object as a collection of EDI-file. 
    Collect edi-files and create an EM object. It sets he properties from 
    audio-magnetotelluric. The two(2) components XY and YX will be set and 
    calculated.Can read MT data instead, however, the full handling transfer 
    function like Tipper and Spectra is not completed. Use other MT software for a long period's data.

.. GENERATED FROM PYTHON SOURCE LINES 65-90

.. code-block:: Python


    from watex.methods.em import EM
    edi_data = load_edis (return_data =True, samples =7) # object from Edi_collection 
    emObjs = EM().fit(edi_data)
    ref=emObjs.getfullfrequency ()  
    ref

    emObjs.freqs_ # # however full frequency can just be fetched using the attribute `freqs_` 

    # get the reference frequency 
    rfreq = emObjs.getreferencefrequency () 
    rfreq

    # Fast process EMAP and AMT data. Tools are used for data sanitizing, 
    # removing noises and filtering. 

    p = EMAP().fit(edi_data) 
    p.window_size =2 
    p.component ='yx'
    rc= p.tma()
    # get the resistivity value of the third frequency  at all stations 
    # >>> p.res2d_[3, :]  

    # get the resistivity value corrected at the third frequency 
    # >>> rc [3, :]







.. GENERATED FROM PYTHON SOURCE LINES 91-92

Plot 1D raw and corrected tensor 

.. GENERATED FROM PYTHON SOURCE LINES 92-94

.. code-block:: Python

    plt.semilogy (np.arange (p.res2d_.shape[1] ), p.res2d_[3, :], '--',
                      np.arange (p.res2d_.shape[1] ), rc[3, :], 'ok--')



.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_001.png
   :alt: plot dc em parameters computing
   :srcset: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    [<matplotlib.lines.Line2D object at 0x7f50a381ee30>, <matplotlib.lines.Line2D object at 0x7f50a381df60>]



.. GENERATED FROM PYTHON SOURCE LINES 95-126

.. code-block:: Python


    # Compute the skew: The conventional asymmetry parameter based on the Z magnitude.

    # p = EMAP().fit(edi_data) 
    # sk,_ = p.skew()
    # sk[0:, ]

    # restore tensor 
    pObjs= EMAP().fit(edi_data)
    # One can specify the frequency buffer like the example below, however 
    # it is not necessary at least there is a specific reason to fix the frequencies 
    # buffer = [1.45000e+04,1.11500e+01]
    zobjs_b =  pObjs.zrestore(
        # buffer = buffer
    ) 
    zobjs_b 

    # control the quality of the EM data 
    #     pobj = EMAP().fit(edi_data)
    #     f = pobj.getfullfrequency ()
    #     # len(f)
    #     # ... 55 # 55 frequencies 
    #     c, = pobj.qc ( tol = .6 ) # mean 60% to consider the data as
    #     # representatives 
    #     c  # the representative rate in the whole EDI- collection
    #      # ... 0.95 # the whole data at all stations is safe to 95%. 
    #     # now check the interpolated frequency 
    #     c, freq_new,  = pobj.qc ( tol=.6 , return_freq =True)
    #     # len(freq_new)
    #     # ... 53  # delete two frequencies 





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    array([<watex.externals.z.Z object at 0x7f50a2a85f60>,
           <watex.externals.z.Z object at 0x7f50a2a86110>,
           <watex.externals.z.Z object at 0x7f50a2a87ca0>,
           <watex.externals.z.Z object at 0x7f50a2a84d30>,
           <watex.externals.z.Z object at 0x7f50a2a85de0>,
           <watex.externals.z.Z object at 0x7f50a2a845b0>,
           <watex.externals.z.Z object at 0x7f50a2a85780>], dtype=object)



.. GENERATED FROM PYTHON SOURCE LINES 127-135

DC-method :mod:`~watex.methods.electrical` 
================================================
A collection of DC-resistivity profiling and sounding classes. 
It reads and computes electrical parameters. Each line or site composes a specific
object and gathers all the attributes of :class:`~.ResistivityProfiling` 
or :class:`~watex.methods.electrical.VerticalSounding`  for easy use. For instance, 
the expected drilling location point  and its 
resistivity value for two survey lines ( line1 and line2) can be fetched 

.. GENERATED FROM PYTHON SOURCE LINES 137-140

DC -Profiling 
----------------
* Get DC -resistivity profiling from the individual Resistivity object 

.. GENERATED FROM PYTHON SOURCE LINES 140-157

.. code-block:: Python


    robj1= ResistivityProfiling(auto=True) # auto detection 
    robj1.utm_zone = '50N'
    data = load_tankesse ()
    robj1.fit(load_tankesse ()) 
    print(robj1.sves_ ) 
    robj2= ResistivityProfiling(auto=True, utm_zone='40S') 
    robj2.fit(load_gbalo()) 
    print(robj2.sves_ ) 
    # read the both objects 
    dcobjs = DCProfiling() 
    dcobjs.fit(robj1, robj2) 
    print(dcobjs.sves_ ) 

    print(dcobjs.line1.sves_) 
    print(dcobjs.line2.sves_ ) 





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    S022
    S036

    dc-erp  :   0%|                                         | 0/2 [00:00<?, ?B/s]
    dc-erp  : 100%|################################| 2/2 [00:00<00:00, 508.62B/s]
    ['S022' 'S036']
    S022
    S036




.. GENERATED FROM PYTHON SOURCE LINES 158-159

Plot conductive zone for line 1 

.. GENERATED FROM PYTHON SOURCE LINES 159-160

.. code-block:: Python

    robj1.plotAnomaly (style =plt_style) 



.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_002.png
   :alt: Plot ERP: SVES = S22
   :srcset: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <AxesSubplot:xlabel='Stations', ylabel='Resistivity (Ω.m)'>



.. GENERATED FROM PYTHON SOURCE LINES 161-162

Plot conductive zones for second lines 

.. GENERATED FROM PYTHON SOURCE LINES 162-163

.. code-block:: Python

    robj2.plotAnomaly ( style = plt_style )



.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_003.png
   :alt: Plot ERP: SVES = S36
   :srcset: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_003.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <AxesSubplot:xlabel='Stations', ylabel='Resistivity (Ω.m)'>



.. GENERATED FROM PYTHON SOURCE LINES 164-165

* Get parameters 

.. GENERATED FROM PYTHON SOURCE LINES 165-167

.. code-block:: Python

    robj1.summary(return_table=True ) 






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>dipole</th>
          <th>longitude</th>
          <th>latitude</th>
          <th>easting</th>
          <th>northing</th>
          <th>sves_resistivity</th>
          <th>power</th>
          <th>magnitude</th>
          <th>shape</th>
          <th>type</th>
          <th>sfi</th>
        </tr>
        <tr>
          <th>station</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>S022</th>
          <td>10</td>
          <td>109</td>
          <td>-81</td>
          <td>382775</td>
          <td>896164</td>
          <td>34</td>
          <td>60</td>
          <td>53</td>
          <td>C</td>
          <td>PC</td>
          <td>0.31221</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 168-171

DC Sounding 
------------- 
* Read single sounding site 

.. GENERATED FROM PYTHON SOURCE LINES 171-177

.. code-block:: Python


    dsobj = DCSounding ()  
    dsobj.search = 30. # start detecting the fracture zone from 30m depth.
    dsobj.fit(load_gbalo(kind ='ves'))
    print(dsobj.ohmic_areas_)
    print(dsobj.site1.fractured_zone_ ) 




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    dc-ves  :   0%|                                         | 0/1 [00:00<?, ?B/s]
    dc-ves  : 100%|################################| 1/1 [00:00<00:00, 138.80B/s]
    [569.03511708]
    [ 28.  32.  36.  40.  45.  50.  55.  60.  70.  80.  90. 100.]




.. GENERATED FROM PYTHON SOURCE LINES 178-179

* read multiple sounding files 

.. GENERATED FROM PYTHON SOURCE LINES 179-199

.. code-block:: Python

    
    dsobj.search = 30 #[ 30, 30, 30, 30] # search values for all sites 
    dsobj.fit(load_semien(index_rhoa= 2 ), 
              load_gbalo(kind ='ves', index_rhoa =2 ,),
              load_boundiali (index_rhoa=1)
              )
    print(dsobj.ohmic_areas_ )  
    print(dsobj.nareas_ ) 

    print(dsobj.survey_names_) 

    print(dsobj.nsites_ ) 

    print(dsobj.site1.ohmic_area_) 

    print(dsobj.data_ ) 
    # you can use the `isnotvalid_` attributes to check the unread data before 
    # calling the site object like 
    print("dsobj.isnotvalid_", dsobj.isnotvalid_) # return empty list if all values passed are correct.





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    dc-ves  :   0%|                                         | 0/3 [00:00<?, ?B/s]
    dc-ves  : 100%|################################| 3/3 [00:00<00:00, 161.95B/s]
    [ 572.84134545 1607.71498659  154.00420841]
    [1. 2. 2.]
    ['site1' 'site2' 'site3']
    3
    572.8413454485877
    [VerticalSounding(AB= 200.0, MN= 20.0, arrangememt= schlumberger, ... , h0= 1.0, strategy= HMCMC, xycoords= (nan, nan)), VerticalSounding(AB= 200.0, MN= 20.0, arrangememt= schlumberger, ... , h0= 1.0, strategy= HMCMC, xycoords= (nan, nan)), VerticalSounding(AB= 200.0, MN= 20.0, arrangememt= schlumberger, ... , h0= 1.0, strategy= HMCMC, xycoords= (nan, nan))]
    dsobj.isnotvalid_ []




.. GENERATED FROM PYTHON SOURCE LINES 200-201

* Plot ohmic area 

.. GENERATED FROM PYTHON SOURCE LINES 201-204

.. code-block:: Python

    ves = VerticalSounding ().fit (dsobj.site3.data_) 
    ves.plotOhmicArea (fbtw=True , style =plt_style)




.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_004.png
   :alt: Vertical Electrical Sounding
   :srcset: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 205-211

Hydrogeology :mod:`~watex.methods.hydro` 
==========================================
Hydrogeological parameters of the aquifer are the essential and crucial basic data 
in the designing and construction progress of geotechnical engineering and 
groundwater dewatering, which is directly related to the reliability of these 
parameters.

.. GENERATED FROM PYTHON SOURCE LINES 211-216

.. code-block:: Python


    # get the logging data 
    h = load_hlogs ()
    print(h.feature_names) 





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ['hole_id', 'depth_top', 'depth_bottom', 'strata_name', 'rock_name', 'layer_thickness', 'resistivity', 'gamma_gamma', 'natural_gamma', 'sp', 'short_distance_gamma', 'well_diameter']




.. GENERATED FROM PYTHON SOURCE LINES 217-218

we can  collect the valid logging data and fit it

.. GENERATED FROM PYTHON SOURCE LINES 218-220

.. code-block:: Python

    log= Logging(kname ='k', zname='depth_top' ).fit(h.frame[h.feature_names])
    print( log.feature_names_in_)  # categorical features should be discarded.




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ['depth_top', 'depth_bottom', 'layer_thickness', 'resistivity', 'gamma_gamma', 'natural_gamma', 'sp', 'short_distance_gamma', 'well_diameter']




.. GENERATED FROM PYTHON SOURCE LINES 221-223

Plot default log using the predictor :math:`X` ( composed of features only)
As an example, we will plot the first five features 

.. GENERATED FROM PYTHON SOURCE LINES 223-225

.. code-block:: Python

    log= Logging(kname ='k', zname='depth_top' ).fit(h.frame[log.feature_names_in_[:5]])
    log.plot ()



.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_005.png
   :alt: depth_bottom, layer_thickness, resistivity, gamma_gamma
   :srcset: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_005.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Logging(zname= depth_top, kname= k, verbose= 0)



.. GENERATED FROM PYTHON SOURCE LINES 226-228

Plot log including the target y 


.. GENERATED FROM PYTHON SOURCE LINES 228-230

.. code-block:: Python

    log.plot (y = h.frame.k , posiy =0 )# first position 




.. image-sg:: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_006.png
   :alt: k, depth_bottom, layer_thickness, resistivity, gamma_gamma
   :srcset: /glr_examples/applications/images/sphx_glr_plot_dc_em_parameters_computing_006.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Logging(zname= depth_top, kname= k, verbose= 0)



.. GENERATED FROM PYTHON SOURCE LINES 231-247

Mixture Learning Strategy (MXS)   
------------------------------- 
The use of machine learning for k-parameter prediction seems an alternative
way to reduce the cost of data collection thereby saving money. However, 
the borehole data comes with a lot of missing k  since the parameter is 
strongly tied to the aquifer after the pumping test. In other words, the 
k-parameter collection is feasible if the layer in the well is an aquifer. 
Unfortunately, predicting some samples of k in a large set of missing data 
remains an issue using the classical supervised learning methods. We, 
therefore, propose an alternative approach called a mixture of learning 
strategy (MXS) to solve these double issues. It entails predicting upstream 
a naïve group of aquifers (NGA) combined with the real values k to 
counterbalance the missing values and yield an optimal prediction score. 
The method, first, implies the K-Means and Hierarchical Agglomerative 
Clustering (HAC) algorithms. K-Means and HAC are used for NGA label 
predicting necessary for the MXS label merging. 

.. GENERATED FROM PYTHON SOURCE LINES 247-266

.. code-block:: Python


    hdata = load_hlogs ().frame 
    # drop the 'remark' columns since there is no valid data 
    hdata.drop (columns ='remark', inplace =True)
    mxs = MXS (kname ='k').fit(hdata)
    # predict the default NGA 
    mxs.predictNGA() # default prediction with n_groups =3 
    # make MXS labels using the default 'k' categorization 
    ymxs=mxs.makeyMXS(categorize_k=True, default_func=True)
    print(mxs.yNGA_ [62:74] ) 

    print(ymxs[62:74])  
    # array([ 1, 22, 22, 22,  3,  1, 22,  1, 22, 22,  1, 22]) 
    # to get the label similarity , need to provide the 
    # the column name of the aquifer group and fit again like 
    mxs = MXS (kname ='k', aqname ='aquifer_group').fit(hdata)
    sim = mxs.labelSimilarity() 
    print(sim ) 
    # [(0, 'II')] # group II and label 0 are very similar




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    [3 2 2 2 1 3 2 3 2 2 3 2]
    [ 3 22 22 22  1  3 22  3 22 22  3 22]
    [(0, 'II')]




.. GENERATED FROM PYTHON SOURCE LINES 267-269

Once the MXS label is created, you can use the supervised training 
strategy for training models. Refer to the model modules (:mod:`~watex.models`)


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 2.654 seconds)


.. _sphx_glr_download_glr_examples_applications_plot_dc_em_parameters_computing.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/watex/watex/0.3.X?urlpath=lab/tree/notebooks/glr_examples/applications/plot_dc_em_parameters_computing.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_dc_em_parameters_computing.ipynb <plot_dc_em_parameters_computing.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_dc_em_parameters_computing.py <plot_dc_em_parameters_computing.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_