Blog of Sara Jakša

How to Show Factor Analysis Loadings in a Better Form

Last time I was doing the factor analysis on the data to try and show how much do I use the Google in my programming. While the code is fine and it prints the required result, it is not exactly in the form that it easy to continue analysing.

The former code outputs it as a vector (rpy2.robjects.vectors.ListVector), but it acts mostly as a string. Since we are doing data analysis, and factor analysis is a statistical method, string is usually not want we want.

I still continued to use the same Big Five personality test found on http://personality-testing.info/_rawdata/.

On the end I decided that something that looked like list of lists seems to be a good idea. I know that R has a data frame, that would be even better, but I am not sure if something like that exist. Also it seems that list of lists can be put into numpy array, one of the structures that are frequently mentioned on different data analysis tutorials.

    import numpy

    def transmuteLoadings(result:)
        results = str(results).split("\n")
        final = []
        for result in results:
            if "[" in result:
                final.append([result[(8-2)+i*8:(8-2)+i*8+6].strip() if result[(8-2)+i*8:(8-2)+i*8+6].strip() else "" for i in range(len(result)//6-2)])
        return numpy.array(final)

This function expects the vector, kind of like the one that was the result of the function that I described on How Many Pieces of Knowledge from Different Places are Hidden in my Code

I only tested it on the above mentioned data, but I don‘t see the reason why it would not work on other data as well.

How Many Pieces of Knowledge from Different Places are Hidden in my Code

For a hobby programmer like me, Google is like a bible for fundamentalists. When they don‘t have an answer for something, they go and find it on bible. When I have a programming problem, I go and find it by googling it.

So I was wondering how many websites do I usually have to access for my to program something. I have a feeling that the number has to be quite high, but I did not know how much.

In order to figure that out, I decide to try and figure out how to program a factor analysis in the python. I wanted to get the loadings of variables for different factors. I continued until I had a working code that could print the loadings in the terminal for analysis. I only counted the websites whose solutions ended up in the code. There were more, but they were dead ends so I did not include them.

For the analysis I used the answers to the Big Five personality questionnaire that I found on http://personality-testing.info/_rawdata/.

This is the final code that I ended up with:

    from sklearn.decomposition import FactorAnalysis
    import numpy
    from rpy2.robjects.packages import importr
    from rpy2.robjects import r, numpy2ri

    data = numpy.genfromtxt('data.csv', delimiter='\t')

    for i in [6,5,4,3,2,1,0]:
        data = numpy.delete(data, i, 1)
    data = numpy.delete(data, 0, 0)

    numpy2ri.activate()
    fit = r.factanal(data, 5, rotation="varimax")
    results = fit.rx('loadings')
    print(results)

I used 7 different websites to code this 12 lines of code. Which means that I needed to check one webpage for ever 1.7 line of code.

Here is the code with websites used above the piece of code that they were used for:

    from sklearn.decomposition import FactorAnalysis
    import numpy

    #http://stackoverflow.com/questions/3518778/how-to-read-csv-into-record-array-in-numpy
    data = numpy.genfromtxt('data.csv', delimiter='\t')

    #http://stackoverflow.com/questions/24898754/delete-dimension-of-array
    #http://docs.scipy.org/doc/numpy/reference/generated/numpy.delete.html
    for i in [6,5,4,3,2,1,0]:
        data = numpy.delete(data, i, 1)
    data = numpy.delete(data, 0, 0)

    #http://stackoverflow.com/questions/25036588/extract-correlation-matrix-from-rs-factanal-via-rpy
    from rpy2.robjects import r, numpy2ri
    numpy2ri.activate()

    #http://blog.yhat.com/posts/rpy2-combing-the-power-of-r-and-python.html
    from rpy2.robjects.packages import importr
    #http://www.statmethods.net/advstats/factor.html
    fit = r.factanal(data, 5, rotation="varimax")
    #http://stackoverflow.com/questions/27575848/how-to-convert-rpy2-listvector-rpy2-robjects-vectors-listvector-to-python
    results = fit.rx('loadings')
    print(results)

I knew that I use internet as a crutch a lot, but these results were surprising. I don‘t really want to believe that I use it that much, but at least in this case the data shows that. Maybe I should rethink my way of programming…

Why Writing Add-ons is in Some Way Simpler

I have recently tried to figure out, how to make a non-obtrusive popup in one of the add-on that I have been writing. I was in luck, as something like that had already been implemented in the program itself.

All I needed to do what figure our when does it appear, find the function that is at work in the same and check what function or group of functions does it call. It ended up being a single function:

    tooltip(string)

When I actually checked, what would I needed to do, if it was not implemented, it look quite a lot more complicated. There was 28 lines of code with mixture of qt signals, timer, changing global variables and even some html and qt element properties to make it look more pretty. Not that complicated as a whole, but compared to a single line, it is.

I believe that similar thing most likely happens when using frameworks or libraries. I don‘t even want to think about what would I do, if I needed to program myself all the statistical element, that came prepackaged in R. Or if I actually needed to program all the website handling that frameworks like sinatra or fat-free framework take care of it themselves.

Even something like choosing a random number would be a pain in the ass without the already implemented solutions like:

    random.randint(a, b)

All of these things make it more possible for the people without a extensive practice or formal education, like me, to still play around and enjoy the working of a code. On the other side, it this making us less willing to actually go into the code and master it? And how many programmers do we need that are the top masters, and how many would simply benefit from being able to string something together?

My Process, When I Need to Understand a Part of Code

I have created some add-ons for some programs. One thing that I need to constantly do is trying to understand a part of the code. Usually it is a part of the code, that does something that I needed done also in my subprogram.

I have so far came up with 2 different way. One is going into the source code, until I understand how it works. It intuitively seems like a way it should be done. But going from a piece of actions, if lucky the function called when I press the button, and then follow the trail until I get to the part I want.

The other was to do it is to take another add-on, that also uses this type of action. Then I hack at the code, deleting all the parts that I don‘t need, until I am left with the so called minimal working sample.

So there was a comment written on one of my add-on pages, and it brought an interesting problem. It wanted it to work in only in certain conditions.

When I did not find the necessary hook in the code quickly, I turned to the second way of doing. A quick git-hub search brought me to the repository for Habit RPG addon. I don‘t even know what it does, but from skimming the short description it did have the part that I needed: how to differentiate between the correct and incorrect answers. It also had the disclaimer that it no longer worked.

So I ended up removing the lines of code, until I was left with the following one:

    from anki.hooks import wrap
    from aqt.reviewer import Reviewer

    def card_answered(self, ease):
        print("This happens with every answer")
        if ease > 1:
            print("This only happens when the answer is correct")

     Reviewer._answerCard = wrap(Reviewer._answerCard, card_answered)

Now I could just use this as it is, or I could go back in to the code to try and understand why it works. Apparently the ease in the function _answerCard() is the button pressed. Since 1 means incorrect, and 2-4 normally mean correct, it provides a good way to differentiate.

Also, the wrap function is there to execute by default the first function and return the second. It can do the reverse with the additional argument: pos == "before".

And that is one way how a self-learned programming person it trying to make sense of the code.

Pizza with Zucchini Dough

Zucchini Pizza

My grandmother gave me the recipe for the zucchini burgers. It apparently only requires the zucchini, rolled oats, eggs and herbs. When I ate it at here place there were delicious, so I decided to make them myself.

But for some reason, they would not hold together at all. They were breaking apart before I could even put them in the pan. So as I was looking at this dough, I decided that I am going to try and make a pizza from it.

Ingredients:

  • zucchini
  • rolled oats
  • eggs
  • toppings (I used cheese, mushrooms, cayenne pepper and basil)

Recipes:

  • Grate the zucchini
  • Add the rolled oats to zucchini
  • Put the mixture on the pan
  • Put the mixed eggs on the top of it
  • Put the toppings on top of it
  • Bake in the oven on 180°C for 15 minutes

Suggestions:

  • I used rolled oats, because they were already mixed in when I came up with the idea. I don't think they are necessary and it would most likely work even without them. They were also the one part that was the worst, so I think it would be an improvement.

Chocolate Pudding Recipe

Chocolate Tapioca Pudding

I have made a horrible mistake during shopping. I was looking for a whipped cream, and I noticed that they have a plant variety. I was already wondering if the one that I usually by is animal or plant based, but it was not written on the carton. So I decided to try the plant one.

But it was way too sweet and totally useless for the normal usage. You know, putting it on the cakes, ice-cream or my favourite, on frozen fruit, especially raspberries.

But since I don't like throwing the food away, I had to find the way of using it somewhere, where it will not be that horrible. And this is where idea for this pudding came from.

Ingredients:

  • tapioca pearls
  • milk (twice the amount of tapioca pearls)
  • water (three times the amount of tapioca pearls)
  • cacao
  • whipped cream

Recipe:

  • Whip the cream
  • Cook the tapioca in mixture of cacao, milk and water until tapioca pearls stop being white (for me it usually takes 20 minutes, but this time it was in a little over 10 minutes)
  • Mix the whipped cream in the pudding
  • Put in the cups and chill

Suggestions:

  • The taste depends on the amount of cacao. The more you put in it, the more likely it is to taste like dark chocolate. But put in too little, and the taste of milk and whipped cream can still be felt. I prefer the dark chocolate to the taste of milk, so I out a lot of it in.
  • Don't put the hot pudding in the refrigerator, as it can break it. Chill it first outside and when at room temperature, put it in the refrigerator

Different Types of Arguing

You have probably noticed by now that everybody don't have the same strategy for arguing. Some people love it, and they are really confrontational, some people are trying to not get into arguments. Some people use facts, while some apply to feelings, and some try to manipulate and deceive.

While perceiving function can influence what kind of data do we use in the arguing, I do believe that it is the judging function that influences what kind of style of arguing are we using.

Introverted feeling types seem like they are trying to avoid any kind of conflict. They do it by disengagement. They usually reply with 'anybody has the right to their own opinion'. An ENFP that I know is really careful with making sure to try and not get into even a heated discussion, let alone argument. A lot of times it comes down to the argument that we have a different taste or opinion or values.

Down below is one example of it:

INTP: I went to the lecture you recommended. I did not like it. ENFP: Who was the lecturer? Person X? INTP: No, it was somebody else. ENFP: If it would be person X (who was organizing it) it would be better. INTP: Even so, they allowed somebody that treated us like we are 10 year old, and they did not even say anything new. ENFP: We do have a different taste.

Extraverted feeling types are using the different tactic to avoid conflict. They are trying to win you over by activating your feelings. I guess a stereotypical politician would be a good example of this. Sometimes this can be seen as a manipulation as well.

Down is one example with a person that I believe uses Fe as primary judging function:

xxFJ: Is there any comments that you want to give to what happened in the last meeting? INTP: I was a little stressed, but I think it went alright. xxFJ: Don't you think that there is something that you have to apologize for? INTP: No? I don't remember doing anything wrong. xxFJ: Don't pretend like you don't know it. (on the end I had to ask what was wrong and apparently I hurt his feelings)

Extraverted thinking types are using the facts from the current reality. They will point out where the ideas don't hold water in the real world. I have a friend that uses Te as primary judging function. He is always pointing out why some ideas are infeasible. He is also likely to bring some random fact to prove his point.

Here is one of the conversations that I had with him a couple of times:

INTP: We really should have a reverse Toastmasters meeting. INTJ: This would not help much, as people would not actually get the accurate evaluations. INTP: They can ask for the evaluations afterwards. INTJ: But you know our members. They never do. INTP: It would be fun. INTJ: It wouldn't.

Introverted thinking types usually go around and try to test the assumptions. That leads to the mostly questioning style that can sometimes remind some people of the Socratic style of learning. I know that a lot of times I like using questions to try and understand things, or simply to try and illuminate the subject matter for other people.

Here is an good example of the Ti in actions:

(There is some news on the television) INTP: But why would people do that? ISTP: Most people are not able to use logic to make decisions. INTP: But what were they thinking? ISTP: If your assumption is true, why did that other thing happen? INTP: Maybe because of X. But if that would be true, shouldn't Y happen?

Sometimes it is good to know what kind of type is the person opposite you. There is no use to try and use logic on the Fi and sometimes even on the Fe types. Just like their natural way of handling the conflict does not work on the Ti and Te types. If you truly want to convince somebody that you are right, maybe giving their argumentative style a try. Even if it will look really pathetic a first couple of times.

Removing all Whitespaces but Normal Spaces with Python and Regex

My sister asked me to help her scrape a website for her summer work. While I was doing is, some of the data was showing a strange behavior, when I tried to put it in the CSV file. It would jump in the next line multiple spaces ahead.

I first tried replacing all the normal suspects, like tabs ("\t"), newlines ("\n") and escapes ("\r") and even double spaces (" ") among other things. But none of that dealt with the problem.

Emil Stenström had published on the Stackoverflow an answer that eventually lead me to my solution. The only problem with his one was, that it also removed the spaces.

So in a fashion of trying to find the quickest solution possible, I used the following code:

    text = text.replace(" ", "ß")
    text = re.sub(r"\s+", "", text, flags=re.UNICODE)
    text = text.replace("ß", " ")

What I did is, that I replaced all the spaces with the letter that I was sure would not appear in the text. Since the text was in Slovenian, this one worked. If I was parsing the website in German, then the choose of letter would not be that good. In that case, using a Slovenian letter like ć would be a much better choice, since it is not likely that it would appear in the text.

I am sure that there exist a more Pythonic way of doing this, but I just was not able to come up with it. Also, google was quite unhelpful in this case.

Where do we get Our Energy?

As soon as people get over the definition of extraversion and introversion as function of socialization and being recluse, they came to the definition of the it as a way the energy move. The introverts get their energy when they are alone and the extraverts get their energy when they are with people.

Looking over my own experience, I believe that the reality is just a tiny bit more complicated. Not every type of extraverted activity is tiring me in the same way, just like not every type of introverted activity is not recharging me on the same way. Actually they are some types of extraverted activities that recharge me and some introverted activities that do no not.

What I like to do for relaxation is programming, cooking and reading. The situations that take the most energy out of me are socializing, or even any kind of dealing with people, adrenaline sports and dealing with bureaucracy.

For me programming is a Ti-Ne type of activity. I analyze the problem with Ti, and then use Ne to try different solutions until one works. Then Ti comes into play again and tries to find something even more elegant. Though maybe I should sometimes test if that solution is actually more efficient, but I just never do it.

Cooking is a Si-Ne-Ti activity. I believe that for most people cooking would have some sort of Se element in it, but it does not in me. First, I am hardly that interested in the taste. I first look at what I have in the pantry, in order to optimize my ingredient usage (Ti). Then I look over my collected knowledge about cooking, wither through my own experience, experience others, something that I have seen on the television or one of the recipes that I collect (Si). I almost never do the recipe the same way, and I always try to find new variation to try (Ne). On the end I try to optimize the usage of the utilities, since I don't like cooking (Ti).

Reading is mostly Si-Ti activity. I have been reading since I was around 4 years old. At least my mother claims that. Reading is what got me through school, indulging the isolation that I experienced, when a student for the other class decided that I raped them. I was ten. For me reading had become like a blanket of content. So sort of like a Nexus (Si). I also use like reading no-fiction, sci-fi and detective stories, where I always can try and make sure that all the pieces of the puzzle fit (Ti).

On the other hand, socializing for me is clearly a Fe activity. Even if a lot of times I try to engage other functions more. Socializing is all about respecting other people personal space and limitations, making sure there are not any social paux-de-fux (Fe). Not only that, these rules are complicated and sometimes have no logical basis or they are situation specific. I know I have to make sure that the ratio of [\frac{other}{people}] have to be as high as possible.

The next one is adrenaline sports, which for me is mostly Se experience. I am scared to go on anything in the amusement parks, because I am scared to death for my own life. I also don't want to jump with the parachute or any other similar types of sports. I just find the physical experience, where I have no control disturbing.

The last one is bureaucracy. Bureaucracy for me is a Te-Si type of activity. Everything is done as it was always done (Si), according to the system that was probably at one time efficient (Te). I have to admit, I am looking over it, and they are becoming more efficient each year. It just takes a lot of time.

Let us take a stack of it. The functions that can potentially recharge me are Ti, Ne, Si. Functions that take the energy away from me are Fe, Se, Te, Si. With that, Fe function is the one that takes the most energy away from me, while the Se function is just something I avoid.

I am an INTP, so for me Ti and Ne functions are my strongest functions, so it makes sense that they are recharging me. They are like a well cared for tool that a person is proficient in using. It is probably equivalent to breathing air.

Beebe had written somewhere that third or eternal child function has the sinus like quality to it. In some situations it is competent, but in the others using it is like pulling teeth. Which probably explains the oddity of my Si, which for me is my third function. Sometimes it recharges me, and sometimes it does not.

Half of the functions that I have noticed that they take energy away from me are shadow functions. Either we can take the explanation that we don't actually use them. In which case they would be like the tool that was never used before. We would not be proficient in using it. Or that they are not differentiated, and are part of the unconscious. In which case it would be like dragging a tool, that was used before, but it is cover is rust. None of which is going to bring the good results.

But the one that tires me the most is my anima or forth function. For me, as an INTP, that is Fe. Beebe said, that this is the function that we inspire after our whole life. Which, if I understood correctly, we can never reach. The other reason, for me more likely, is that it is the opposite of the dominant function. It would be like person used to using black, suddenly being forced to use white. It would confuse a person and there would be none of the proficiency there.

So what is the conclusion. I believe that the functions most likely to recharge us are the dominant and auxiliary functions. The one that takes the most energy away from us is the firth function. And the rest are somewhere in between.

Cottage Cheese Rolled Dumplings

Cottage cheese struklji

A type of cottage cheese rolled dumplings is a traditional food in Slovenia. It is actually the one that I usually make if asked to cook something from my own country. The reason is simple. I am vegetarian, I don't like fermented food and usually don't want to spend up to 10 hours in the kitchen for just one dish. The number of remaining dished that could be considered traditional in Slovenia is small, and this is one of them.

There exist quite a couple of variants. If they are to be used as a side dish, then they are usually made sour. But they can be made sweet and used as a dessert.

Ingredients:

  • flour
  • salt
  • water
  • eggs
  • oil
  • cottage cheese
  • sour cream
  • sugar (not necessary)

Recipe:

  • Mix flour, salt, water, egg and oil to make a dough
  • Let the dough sit for at least an hour
  • Mix cottage cheese, sour cream. Add sugar if making a sweet variety
  • Roll out the dough. Put the cottage cheese mixture on it and roll
  • Cook in boiling water for at least 20 minutes
  • Cool down before serving

They go along together with with with any kind of sauce or breadcrumbs. Breadcrumbs are heated in the pun in the melted butter.