watex.utils.baseutils.array2hdf5(filename, /, arr=None, dataname='data', task='store', as_frame=Ellipsis, columns=None)[source]#

Load or write array to hdf5

Parameters

  • arr (Arraylike ( m_samples, n_features)) – Data to load or write

  • filename (str,) – Hdf5 disk file name whether to write or to load

  • task (str, {"store", "load", default='store'}) – Action to perform. user can use [‘write’|’store’] interchnageably. Both does the same task.

  • as_frame (bool, default=False) – Concert loaded array to data frame. Columns can be supplied to construct the datafame.

  • columns (List, Optional) – Columns used to construct the dataframe. When its given, it must be consistent with the shape of the arr along axis 1

Returns

None| data

Return type

ArrayLike or pd.DataFrame

Examples

>>> import numpy as np
>>> from watex.utils.baseutils import array2hdf5
>>> data = np.random.randn (100, 27 )
>>> array2hdf5 ('test.h5', data   )
>>> load_data = array2hdf5 ( 'test.h5', data, task ='load')
>>> load_data.shape
Out[177]: (100, 27)
watex.utils.baseutils.check_file_exists(package, resource)[source]#

Check if a file exists in a package’s directory with importlib.resources.

Parameters

  • package – The package containing the resource.

  • resource – The resource (file) to check.

Returns

Boolean indicating if the resource exists.

Example
>>> from watex.utils.baseutils import check_file_exists
>>> package_name = 'watex.datasets.data'  # Replace with your package name
>>> file_name = 'h.h5'    # Replace with your file name

>>> file_exists = check_file_exists(package_name, file_name)
>>> print(f"File exists: {file_exists}")
watex.utils.baseutils.download_file(url, local_filename, dstpath=None)[source]#

download a remote file.

Parameters

  • url (str,) – Url to where the file is stored.

  • loadl_filename (str,) – Name of the local file

  • dstpath (Optional) – The destination path to save the downloaded file.

Returns

None if the dstpath is supplied and local_filename otherwise.

Return type

None, local_filename

Example

>>> from watex.utils.baseutils import download_file
>>> url = 'https://raw.githubusercontent.com/WEgeophysics/watex/master/watex/datasets/data/h.h5'
>>> local_filename = 'h.h5'
>>> download_file(url, local_filename, test_directory)
watex.utils.baseutils.download_file2(url, local_filename, dstpath=None)[source]#

Download remote file with a bar progression.

Parameters

  • url (str,) – Url to where the file is stored.

  • loadl_filename (str,) – Name of the local file

  • dstpath (Optional) – The destination path to save the downloaded file.

Returns

None if the dstpath is supplied and local_filename otherwise.

Return type

None, local_filename

Example

>>> from watex.utils.baseutils import download_file2
>>> url = 'https://raw.githubusercontent.com/WEgeophysics/watex/master/watex/datasets/data/h.h5'
>>> local_filename = 'h.h5'
>>> download_file(url, local_filename)
watex.utils.baseutils.get_remote_data(rfile, /, savepath=None, raise_exception=True)[source]#

Try to retrieve data from remote.

Parameters

  • rfile (str or PathLike-object) – Full path to the remote file. It can be the path to the repository root toward the file name. For instance, to retrieve the file 'AGSO.csv' which is located in watex/etc/ directory then the full path should be 'watex/etc/AGSO.csv'

  • savepath (str, optional) – Full path to place where to downloaded files should be located. If None data is saved to the current directory.

  • raise_exception (bool, default=True) – raise exception if connection failed.

Returns

status –

False for failure and True otherwise i.e. successfully

downloaded.

Return type

bool,

watex.utils.baseutils.lowertify(*values, strip=True, return_origin=Ellipsis)[source]#

Strip and convert value to lowercase.

Parameters

value – str , value to convert

Returns

value in lowercase and original value.

Example
>>> from watex.utils.baseutils import lowertify
>>> lowertify ( 'KIND')
Out[19]: ('kind',)
>>> lowertify ( "KIND", return_origin =True )
Out[20]: (('kind', 'KIND'),)
>>> lowertify ( "args1", 120 , 'ArG3')
Out[21]: ('args1', '120', 'arg3')
>>> lowertify ( "args1", 120 , 'ArG3', return_origin =True )
Out[22]: (('args1', 'args1'), ('120', 120), ('arg3', 'ArG3'))
>>> (kind, kind0) , ( task, task0 ) = lowertify(
    "KIND", "task ", return_origin =True )
>>> kind, kind0, task, task0
Out[23]: ('kind', 'KIND', 'task', 'task ')
watex.utils.baseutils.move_file_to_directory(file_path, directory)[source]#

Move file to a directory.

Create a directory if not exists.

Parameters

  • file_path (str,) – Path to the local file

  • directory (str,) – Path to locate the directory.

Example

>>> from watex.utils.baseutils import move_file_to_directory
>>> file_path = 'path/to/your/file.txt'  # Replace with your file's path
>>> directory = 'path/to/your/directory'  # Replace with your directory's path
>>> move_file_to_directory(file_path, directory)
watex.utils.baseutils.request_data(url, /, task='get', data=None, as_json=Ellipsis, as_text=Ellipsis, stream=Ellipsis, raise_status=Ellipsis, save2file=Ellipsis, filename=None, **kws)[source]#

Fetch remotely data

Request data remotely https://docs.python-requests.org/en/latest/user/quickstart/#raw-response-content

r = requests.get(’https://api.github.com/user’, auth=(‘user’, ‘pass’)) r.status_code 200 r.headers[‘content-type’] ‘application/json; charset=utf8’ r.encoding ‘utf-8’ r.text ‘{“type”:”User”…’ r.json() {‘private_gists’: 419, ‘total_private_repos’: 77, …}

watex.utils.baseutils.save_or_load(fname, /, arr=None, task='save', format='.txt', compressed=Ellipsis, comments='#', delimiter=None, **kws)[source]#

Save or load Numpy array.

Parameters

  • fname (file, str, or pathlib.Path) – File or filename to which the data is saved. - >.npy , .npz: If file is a file-object, then the filename is unchanged. If file is a string or Path, a .npy extension will be appended to the filename if it does not already have one. - >.txt: If the filename ends in .gz, the file is automatically saved in compressed gzip format. loadtxt understands gzipped files transparently.

  • arr (1D or 2D array_like) – Data to be saved to a text, npy or npz file.

  • task (str {"load", "save"}) – Action to perform. “Save” for storing file into the format “.txt”, “npy”, “.npz”. “load” for loading the data from storing files.

  • format (str {".txt", ".npy", ".npz"}) – The kind of format to save and load. Note that when loading the compressed data saved into npz format, it does not return systematically the array rather than np.lib.npyio.NpzFile files. Use either files attributes to get the list of registered files or f attribute dot the data name to get the loaded data set.

  • compressed (bool, default=False) – Compressed the file especially when file format is set to .npz.

  • comments (str or sequence of str or None, default='#') –

    The characters or list of characters used to indicate the start

    of a comment. None implies no comments. For backwards compatibility, byte strings will be decoded as ‘latin1’. This is useful when fname is in txt format.

    delimiter: str, optional

    The character used to separate the values. For backwards compatibility, byte strings will be decoded as ‘latin1’. The default is whitespace.

  • kws (np.save ,np.savetext, np.load , np.loadtxt) – Additional keywords arguments for saving and loading data.

Returns

None| data

Return type

ArrayLike

Examples

>>> import numpy as np
>>> from watex.utils.baseutils import save_or_load
>>> data = np.random.randn (2, 7)
>>> # save to txt
>>> save_or_load ( "test.txt" , data)
>>> save_or_load ( "test",  data, format='.npy')
>>> save_or_load ( "test",  data, format='.npz')
>>> save_or_load ( "test_compressed",  data, format='.npz', compressed=True )
>>> # load files
>>> save_or_load ( "test.txt", task ='load')
Out[36]:
array([[ 0.69265852,  0.67829574,  2.09023489, -2.34162127,  0.48689125,
        -0.04790965,  1.36510779],
       [-1.38349568,  0.63050939,  0.81771051,  0.55093818, -0.43066737,
        -0.59276321, -0.80709192]])
>>> save_or_load ( "test.npy", task ='load')
Out[39]: array([-2.34162127,  0.55093818])
>>> save_or_load ( "test.npz", task ='load')
<numpy.lib.npyio.NpzFile at 0x1b0821870a0>
>>> npzo = save_or_load ( "test.npz", task ='load')
>>> npzo.files
Out[44]: ['arr_0']
>>> npzo.f.arr_0
Out[45]:
array([[ 0.69265852,  0.67829574,  2.09023489, -2.34162127,  0.48689125,
        -0.04790965,  1.36510779],
       [-1.38349568,  0.63050939,  0.81771051,  0.55093818, -0.43066737,
        -0.59276321, -0.80709192]])
>>> save_or_load ( "test_compressed.npz", task ='load')
...