SMath Studio Forum
»
SMath Studio
»
Questions
»
Time for calculating the variance() & stddev() from csv file
Rank: Newbie
Groups: Registered
Joined: 04/04/2024(UTC) Posts: 2
|
Hi SMath community, I would like to calculate the variance() and StdDev() of the second column "V(3v3)" of the attached file with 50k datapoints. Unfortunately the calculation takes forever (aborted it after 30min). The file is an export of a circuit simulation result (gaussian tolerance distribution in LTSpice). Originally it is in .txt-Format and it seems that I have to convert it in excel to .csv-Format before I can use is. I attached both, the original LTSpice export (.txt) and the converted one from Excel (.csv). How can I handle this in sMath? Archiv.zip (586kb) downloaded 8 time(s).Thanks in advance, Marc Edited by user 04 April 2024 02:21:02(UTC)
| Reason: Not specified
|
|
|
|
Rank: Advanced Member Groups: Registered, Advanced Member Joined: 13/01/2012(UTC) Posts: 2,648 Location: Italy Was thanked: 1331 time(s) in 876 post(s)
|
With import file functions you might need to set the optimization to numeric to speed up the loading. |
If you like my plugins consider to support SMath Studio buying a plan; to offer me a coffee: paypal.me/dcprojects |
|
|
|
Rank: Newbie
Groups: Registered
Joined: 04/04/2024(UTC) Posts: 2
|
Optimizations were all set to numeric - did not help. What helped was that I substituted the statistic function variance() and StdDev with the original/native formula for the variance and standard deviation. This speeds it up to around 1s. Don´t know what happens inside the functions of the statistic plugin but it is very slow. Thanks
|
|
|
|
Rank: Member Groups: Registered
Joined: 21/11/2023(UTC) Posts: 24 Was thanked: 2 time(s) in 2 post(s)
|
I have noticed the same thing; the built-in function for standard deviation takes forever but the calculation runs pretty quickly if I just code in the formula in the worksheet.
|
|
|
|
Rank: Advanced Member Groups: Registered, Advanced Member Joined: 13/01/2012(UTC) Posts: 2,648 Location: Italy Was thanked: 1331 time(s) in 876 post(s)
|
Confirmed, the bottleneck seems to be in the final ouput processing to return the result from the equation [f.e. 1/{n-1}*sum(((el(X,i)-mean)^2),i,1,n)) for StdDev]; no idea not sure what could be the cause... |
If you like my plugins consider to support SMath Studio buying a plan; to offer me a coffee: paypal.me/dcprojects |
|
|
|
SMath Studio Forum
»
SMath Studio
»
Questions
»
Time for calculating the variance() & stddev() from csv file
Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.