Blog of Sara Jakša

How Accurate are the Tests Measuring Personality Differences?

There are a lot of tests measuring differences between people. From The Big Five, Intelligence, MBTI, Jung's Functions, Socionics, DISC, and many, many, many others. I like solving these different tests, wanting to see what they are going to be saying about me.

But not all the tests are equally good. For example, if I would be doing an MBTI test, I would usually get an INTP as a result, but sometimes I would get INTJ, and on one memorable occasion, I got INFP. Since I know, that no test can be 100% accurate, I am alright with getting some mistakes from time to time. Considering, that I am an INTP, getting the INTJ or INFP is not that far off. If I remember correctly, I think that for research, a test would need to be accurate at least 70% of the time, but for an individual assessment, under which doing tests to find out more about it, it would need to be 90% accurate. At least when it comes to the professional tests, online tests probably don't really check this.

But there are some other tests, where I am not so sure, how accurate they are. One of the ways, how to measure this, is to do a test-retest. This means, taking the test at one point, and then repeating it at another point. In research, a huge amount of people would need to take the test at two different times. Since most of the tests that I do are not in research capacity, I can normally only do my own test-retest.

As I was a part of the Erasmus+, program, I needed to take the memo test. They measure the 6 personality tests connected to employability. Since I participated in the program more than once, I had to do the test more than once. So, I have the same test done twice, once in July 2017 and another in September 2017. Considering the time elapsed between them, my personality should not change a lot. The results are in percentiles, compared to all the other people, who did the same assessment.

Factor July 2017 September 2017
Confidence 65% 85%
Curiosity 20% 25%
Decisiveness 20% 55%
Problem-Solving 35% 45%
Self-assessment 20% 35%
Tolerance 60% 85%
Employability 25% 55%

I also added the description of the factors, in case anybody is interested:

Factor Description
Confidence High values on this factor point to a high degree of self-sufficiency and a strong conviction of one’s own abilities – aspects that may positively impact career success. Individuals with high values for this factor may however also be inflexible and set in their ways. Low values show doubt about one’s own abilities and perseverance, which might be grounded in negative experience or insecurity.
Curiosity High values of this factor indicate that a person is not only open to new experiences but actively seeks them out. This also applies to new personal challenges. Low values hint at an altogether more reluctant attitude towards new experiences and a greater appreciation of what is familiar.
Decisiveness High values point to an active and decisive individual, who may have a critical attitude toward their respective field. Low values suggest that the individual is more likely to reconsider his or her decisions to accommodate the opinions of others.
Problem-Solving High values reflect a “problem-solver” who does not like to delve into the insoluble aspects of a task but rather focuses on the doable and likes a challenge. Such individuals may either be very pragmatic in their approach to education, considering it to solve practical problems, or else very theory-oriented in that they are attracted to problem-solving as an academic exercise. Low values reflect an individual who is aware of problems or the problematic aspects of a situation and might be more concerned with identifying the problem than with solving it. Accordingly, such an individual would be less goal-oriented and may have an altogether less future-oriented perspective on things.
Self-assessment High values for this factor indicate that a person knows his or her strengths and weaknesses. This capacity for self-assessment not only leads to a more relaxed relationship with other people or new demands, but also might prevent disappointment with their job or higher education institution. Low values, on the other hand, suggest an altogether higher stress level that can be caused by a misjudgment of one’s own abilities, accompanied by difficulty understanding the demands and requirements of a work position or study program.
Tolerance High values of this factor mean that a person is capable of tolerating the behavior and values of other people without compromising his or her own values. Low values mean that a person feels very uncomfortable if confronted with other people’s different values and ways of life. Such individuals may espouse a more traditional view of things, based on their own perspective and experience as influenced by family, society and established norms and values. Deviation from what is conceived of as “normal” is perceived as threatening or at least discomforting.

As you can see from my results, they changed quite a lot in 2 months. But the structure stayed the same. My advantages are confidence and tolerance. Since I only have 1 data point, it is hard to tell, if I am part of a trend or an outlier, but I would be a bit careful at interpreting these results.

Of course, before we even come to the test-retest problem, we first need to make sure the test is measuring what it is supposed to measure. And this is something, that I am not that sure of in one of the tests, that I have seen previously.

Toastmasters clubs have recently implemented a change in their education program. One, in one of the projects, there is a project about the communication style. Which, alright, I can see how this could be useful for people practicing public speaking. They describe 4 different communication styles. Down below, I am giving a snippet from each description of a communication style.

Style Description
Direct This style is decisive, competitive, independent, and confident. Because the person with a Direct communication style is focused, results-oriented, ambitious, goal-oriented, and driven, others may perceive her as strong-willed or demanding.
Initiating This style is sociable, enthusiastic, energetic, spontaneous, and fun-loving. Due to the gregarious nature of the person with an Initiating communication style, he may be perceived as someone who talks more than listens.
Supportive This style is calm, steady, approachable, sincere, and gentle. Because the person with a Supportive communication style dislikes change, she may appear indecisive. Due to her active listening skills, others see her as cooperative, dependable, and loyal.
Analytical This style is precise, exact, analytical, and logical. Because the person with an Analytical communication style is systematic and task-oriented, he is sometimes perceived as a perfectionist.

Sure, another four-quadrant system. They are not clear on what the axis are, but I guess it does help to put it in a sort of a structure. I mean, the theory by itself, I can see how it can be helpful to somebody, that never thought about individual differences in a structured way.

The problem is with the assessment. There was a test, that was supposed to help people see, which style is most like them. I will post the questions below, to see, if anybody can see the problem:

Question Direct Initiating Supportive Analytical
I like to be seen as decisive sociable patient cautious
When collaborating with others, I am competitive spontaneous cooperative precise
When it comes to sharing feelings with others, I am reserved self-assured sympathetic private
Others would describe me as result-oriented fun-loving gentle disciplined
I like my appearance to be business-like stylish casual analytical
I want support for my goals ideas feelings thoughts
When working on a project, I am ambitious enthusiastic purposeful persistent
Some may perceive me negatively as strong-willed impulsive indecisive perfectionist
As a leader, I would be perceived as focused risk-taker dependable problem-solver
When placed in a new situation, I am fearless optimistic friendly conscientious
My perfect occupation would give me a sense of accomplishment enjoyment acceptance accuracy
My favorite way to work is by delegating networking brainstorming independently

Do you see it. To me, these questions don't really measure the communication style, since some of them are not really about communication at all. It seems more like, they saw that there were some things that correlate (or intuitively seem to be something to correlate) and they added them as questions. But I am not sure, that if I put them in the factor analysis, that I will get the results that I want.

This is another problem. For the test to be good for something, it needs to measure what it says it measures. And I am not sure this one does. For one thing, I don't think the results of this test helped me clarify anything. Any I am not the only one. Model itself was a bit helpful, but the test was confusion as hell. I mean, I got initiating as my highest one. Sure, my communication style can sometimes be like that (though, I don't think it is my most frequent one), but the rest of the description, that was in the book... not me at all.

I mean, this is what they recommended for my communication type:

Initiating communicators value interacting with others and sharing stories. Allow time for socializing at the beginning of meetings and create a friendly, non-threatening environment. Provide time for them to express their feelings and opinions.

Which is just so... wrong. I mean, put me in the environment like that, and I will feel uncomfortable. I don't like people trying to serve me. You know, all these nice people, that try to make sure that everybody around them is taken care of, and they get out of their way to make sure everybody is included, and all their needs met. Aren't they annoying. But this description sort of reminds me of them.

So, in this case, I would take it with pinch of salt. This is in opposite to, for example, astrology, where I can be sure that it does not really apply to me. I mean, I am cancer, so I was supposed to be connected to the family (this is maybe even true), appreciate my home (this is why I lived in 5 countries so far, right?), be a nice person (which is why some people are intuitively afraid of me) and cry a lot (if I cry a lot, then most people never do). Astrology got me so wrong, that I don't mind ignoring it out-right for now.

What to take seriously and what not in the end? If you don't want to read a lot of scientific articles for each test, what I normally do is see how useful this for me is. Can this help me explain something about myself or somebody else? Than if might be worth a look. If it got me wrong, then maybe leave it out for now.