On the Hausdorff Distance Between the Heaviside Function and Some Transmuted Activation Functions
Nikolay Kyurkchiev^{1}, Anton Iliev^{1, 2}
^{1}Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Sofia, Bulgaria
^{2}Faculty of Mathematics and Informatics, Paisii Hilendarski University of Plovdiv, Plovdiv, Bulgaria
Email address:
To cite this article:
Nikolay Kyurkchiev, Anton Iliev. On the Hausdorff Distance Between the Heaviside Function and Some Transmuted Activation Functions. Mathematical Modelling and Applications. Vol. 1, No. 1, 2016, pp. 8-12. doi: 10.11648/j.mma.20160101.12
Received: August 18, 2016; Accepted: October 12, 2016; Published: October 14, 2016
Abstract: In this paper we study the one-sided Hausdorff distance between the Heaviside function and some transmuted activation functions. Precise upper and lower bounds for the Hausdorff distance have been obtained. Numerical examples are presented throughout the paper using the computer algebra system MATHEMATICA. The results can be successfully used in the field of applied insurance mathematics.
Keywords: Transmuted Activation Functions, Heaviside Function, Hausdorff Distance, Upper and Lower Bounds, Squashing Function
1. Introduction and Preliminaries
In this paper we discuss some computational and approximation issues related to several classes of transmuted activation functions.
Sigmoidal functions find numerous applications in various fields related to Life sciences, chemistry, physics, artificial intelligence, signal processes, pattern recognition, machine learning, demography, economics, probability, financial mathematics, statistics, fuzzy set theory, etc.
The following are common examples of activation functions [2]:
Sigmoidal functions (are also known as "activation functions") find multiple applications to neural networks [9], [10], [11], [12], [13], [2].
Definition 1. Define the Heaviside step function as:
(1)
About approximation of the Heaviside step function by some cumulative distribution functions, see [14].
Definition 2. The arctan activation function (sigmoidal Cauchy cumulative distribution function) is defined for by [1]:
(2)
Definition 3. A random variable is said to have a transmuted distribution if its cumulative distribution function (cdf) is given by [3], [4]:
(3)
where is the cdf of the base distribution.
Definition 4. The Hausdorff distance between two interval functions on , is the distance between their completed graphs and considered as closed subsets of [5], [6], [7].
More precisely, we have
(4)
wherein is any norm in , e. g. the maximum norm ; hence the distance between the points , in is .
In this work we prove estimates for the one–sided Hausdorff approximation of the Heaviside function by transmuted Cauchy function.
The results are relevant for applied insurance mathematics [14].
The transmuted Cauchy distribution function has been used also in the analysis of extreme values.
Let us point out that the Hausdorff distance is a natural measuring criteria for the approximation of bounded discontinuous functions [7], [8].
Definition 5. Consider the following transmuted Cauchy function
(5)
where .
Figure 1. Approximation of the Heaviside function by transmuted Cauchy function for the following data: , ; Hausdorff distance .
Figure 2. Approximation of the Heaviside function by transmuted Cauchy function for the following data: , ; Hausdorff distance .
Approximation of the Heaviside function by transmuted Cauchy function for specific values of , and can be seen on Fig. 1 and Fig. 2.
2. Main Results
We study the Hausdorff approximation of the Heaviside function by the transmuted Cauchy function (5) and look for an expression for the error of the best one–sided approximation.
The following Theorem gives upper and lower bounds for :
Theorem 2.1 For the Hausdorff distance between the function and the transmuted Cauchy function (5) the following inequalities hold for and :
(6)
Proof. We need to express in terms of and .
The Hausdorff distance satisfies the relation
(7)
Consider the function
In addition and is strictly monotonically increasing.
By means of Taylor expansion we obtain
Hence approximates with as (see Fig. 3).
Further, for and we have
This completes the proof of the theorem.
Figure 3. The functions and for , .
On Fig. 3 can be seen a good approximation of the function .
Some computational examples using relations (6) are presented in Table 1.
Table 1 shows that at fixed and decreasing values of parameter , Hausdorff distance is reduced. The reader can visualize that declining trend.
Table 1. Bounds for computed by (6) for various , .
λ | b | d1 | dr | d from (7) |
0.18 | 0.2 | 0.135494 | 0.270829 | 0.251167 |
0.001 | 0.2 | 0.128663 | 0.263831 | 0.228837 |
0.1 | 0.1 | 0.0826817 | 0.20615 | 0.185118 |
0.18 | 0.1 | 0.0850281 | 0.209575 | 0.198145 |
The last column of Table 1 contains the values of computed by solving the nonlinear equation (7).
The resulting "fork" to the root and particularly the assessment above are satisfactory.
3. Conclusion
New estimates for the Hausdorff distance between an interval Heviside step function and its best approximating transmuted Cauchy function are obtained.
The assessment of the value of the best Hausdorff approximation is in close contact with many interesting problems in the field of mathematics in insurance - an assessment of the value of the achieved liability insurance - unfortunately at a slow pace, which is in unison with the law of diminishing marginal returns.
In the present paper we do not consider transmuted cumulative distribution functions, such as the
Raised–Cosine transmuted function:
Half–Cauchy transmuted function:
Kumaraswamy–Half–Cauchy transmuted function:
Hyperbolic–Secant transmuted function:
where .
Based on the methodology proposed in the present note, the reader may formulate the corresponding approximation problems on his/her own.
The Hausdorff approximation of the interval step function by the logistic and other sigmoid functions is discussed from various approximation, computational and modelling aspects in [15]–[25].
4. Remarks
Definition 6. The cut function on is defined for by
Special case 1. For we obtain a cut function on the interval :
Special case 2. For we obtain the cut function on :
Definition 7. The smooth sigmoid raised-cosine cumulative distribution function (SRCCDF.cdf) is defined for , , by:
(8)
Approximation of the cut function by (SRCCDF.cdf) .
Note that the function (8) has an inflection at its "centre" and its slope at is equal to .
On Fig. 4 can be seen the cut and (SRCCDF.cdf) for specific concrete values of and .
The following Theorem is valid.
Theorem. The function with is the SRCCDF.cdf function of best Hausdorff one-side approximation to function and for H-distance the following holds for :
The proof of the Theorem follows the ideas given in this paper and will be omitted.
Figure 4. The cut and (SRCCDF.cdf) for , .
Considering this interesting result, the reader may formulate and explore the generalized transmuted cumulative distribution functions of the above functions - and .
As an example,
References