Accuracy checking and hallucinations

General
1

Hi all,

My university based research team has been using Julius for running various regression models and statistical comparison testing. It ran excellent binary logistic regression and nonparametric chi square models with ease. We’re also using it for qualitative data analysis for theme generation. For quantitative models, we’ve been comparing results to those generated from SPSS or STATA. For qualitative analyses, we check results against coding themes by hand. Things seemed to check out in terms of accuracy. I know Julius uses ChatGPT and Claude and I’m a bit familiar with their general accuracy rates knowing that even a few small errors could undermine research results. Just wondering if anyone else has systematically checked the accuracy of results generated from Julius for quality control?

BTW, love Julius so far!

Thanks!
Andrew

4 Likes
2

Hi all. Yes I’m also wondering about the accuracy. I find Julius helpful in developing outcomes based on non parametric analysis. Although I’ve trying to determine it’s accuracy based on the more traditional tools Stata and R. Any thoughts out there?

1 Like
3

Hi,
Once I collected survey data and experimented with Julius. The reliability index (cronbach alpha) was quite good. This indicates the internal consistency of the variable. which might take for spss a good amount of time. I am trying to run some other tests with julius, will share the result very soon. It also generates good topic consisting of depending and independent variables.

1 Like