DEV Community

Caleb Zhao
Caleb Zhao

Posted on

Incorrect calculations: sec(x) near x=k*π+π/2

In this blog post, we explore incorrect calculations of the secant function sec(x) in MATLAB near numbers around kπ+π2. k\,\pi+\frac{\pi}{2}.

Example 1. Given that x1=0.1099557429e23π+π2 x_1=0.1099557429\textup{e}2 \approx 3\,\pi+\frac{\pi}{2} and
x2=0.8011061266654e225π+π2,x_2=0.8011061266654\textup{e}2 \approx 25\,\pi+\frac{\pi}{2}\,, calculate sec(x1) \sec(x_1) and sec(x2) \sec(x_2) in MATLAB.

Using MATLAB, we computed the values of sec(x1) \sec(x_1) and sec(x2). \sec(x_2). The result is shown in the following screenshot.

MATLAB outputFrom the above screenshot, it can be seen that the outputs from MATLAB are
4.105555334642597e+084.10555\red{5334642597}\textup{e}+08 and 3.580150224915013e+123.\red{580150224915013}\textup{e}+12 , respectively.

However, the correct values with 16 significant digits are 0.4105556037464873e9 and 0.3670813182326778e13, respectively (as provided by ISRealsoft).

Thus, the output of MATLAB contains only 6 and 1 correct digits, with error rates of (16-6)/16 = 62.5% and (16-1)/16 = 93.75% respectively for the significant figures.

previous \,\quad\quad\quad\quad\hspace{7.2cm} next

Top comments (0)