watex.utils.funcutils.fillNaN#

watex.utils.funcutils.fillNaN(arr, method='ff')[source]#

Most efficient way to back/forward-fill NaN values in numpy array.

Parameters:

  • arr (ndarray) – Array containing NaN values to be filled

  • method (str) – Method for filling. Can be forward fill ff or backward fill bf`. or both for the two methods. Default is ff.

Return type:

new array filled.

Notes

When NaN value is framed between two valid numbers, ff and bf performs well the filling operations. However, when the array is ended by multiple NaN values, the ff is recommended. At the opposite the bf is the method suggested. The ``both``argument does the both tasks at the expense of the computation cost.

Examples

>>> import numpy as np
>>> from from watex.utils.funcutils import fillNaN
>>> arr2d = np.random.randn(7, 3)
>>> # change some value into NaN
>>> arr2d[[0, 2, 3, 3 ],[0, 2,1, 2]]= np.nan
>>> arr2d
... array([[        nan, -0.74636104,  1.12731613],
       [ 0.48178017, -0.18593812, -0.67673698],
       [ 0.17143421, -2.15184895,         nan],
       [-0.6839212 ,         nan,         nan]])
>>> fillNaN (arr2d)
... array([[        nan, -0.74636104,  1.12731613],
       [ 0.48178017, -0.18593812, -0.67673698],
       [ 0.17143421, -2.15184895, -2.15184895],
       [-0.6839212 , -0.6839212 , -0.6839212 ]])
>>> fillNaN(arr2d, 'bf')
... array([[-0.74636104, -0.74636104,  1.12731613],
       [ 0.48178017, -0.18593812, -0.67673698],
       [ 0.17143421, -2.15184895,         nan],
       [-0.6839212 ,         nan,         nan]])
>>> fillNaN (arr2d, 'both')
... array([[-0.74636104, -0.74636104,  1.12731613],
       [ 0.48178017, -0.18593812, -0.67673698],
       [ 0.17143421, -2.15184895, -2.15184895],
       [-0.6839212 , -0.6839212 , -0.6839212 ]])

References

Some function below are edited by the authors in pyQuestion.com website. There are other way more efficient to perform this task by calling the module Numba to accelerate the computation time. However, at the time this script is writen (August 17th, 2022) , Numba works with Numpy version 1.21. The latter is older than the one used in for writting this package (1.22.3 ).

For furher details, one can refer to the following link: https://pyquestions.com/most-efficient-way-to-forward-fill-nan-values-in-numpy-array