2016-03-20

Cogito, ergo sum

Some days ago I wrote a post on AlphaGo, DeepMind and the GO competition. In the post I urged DeepMind to go for something much harder StarCraft. This was not a random selection, my youngest son happened to be one of Europe's best StarCraft player some years ago. First I dismissed his obsession with the game as just unhealthy waste of time. But after watching him play a few times I changed my mind. I realised StarCraft is an extremely complex game. I still remember the son preparing for a game. ‘I must be in a good mood and have slept well, otherwise I will loose”, “I need to be alert”, he warmed up as a top athlete, with finger practices for the dexterity. StarCraft openings are slow you don’t see much activity it’s more strategy you position for the final kill in the end play where the speed of keystrokes and mouse moves often are insane. “One erroneous move and you’re lost”.
“The best players are not particularly fast, it’s a strategy game much more complex than Chess” my son said. He spend more time studying other players and reading game theory. He tried to study old games of upcoming opponents to understand how they play and finding ways to outsmart them.


I think StarCraft will be a hard nut to crack for DeepMind. They or some other will finally defeat humans in a fair game of StarCraft of that I’m am sure. But will that program contain true intelligence I’m not so sure. It will definitely not be ‘human intelligent’, we may eventually construct ‘human intelligence’ or even ‘super human intelligence’ but that will be 20 years from now for a long time to come, of that I’m pretty certain.
Lately I seen people claiming the Turing test is wrong, since a program can pass the test without having all human characteristics of intelligence. Turing himself was open to ‘alien’ artificial intelligence. One day in a distant future we may see AI smarter than human. But then I’m gone and you too. Actually all of mankind will be annihilated.  

Starting with D again

I started for the X time to study the D language, for some reason I like the language but cannot muster enough energy to do more than a few HW programs. This time I will devote some of the Easter to the language and see if I can connect to SAP just for the hell of it. But just install D caused me some problems Symantec does not like D, first installing the langauage gave me a severe warning:
  


Then installing the project build tool DUB made Symantec annoyed:
 


Symantec simply removed the DUB installer to protect me. However after some fiddling around with a zipped version and installed openRAR via Windows app store I finally got DUB installed.

Now I can start creating the HW program D version for the zillionth time, it remains to be seen if I can make it further this time.

2016-03-13

DeepMind GO for StarCraft.

I just watched a fascinating game of GO where the world champion Lee Sedol defeated the Google DeepMind program for the first time.
DeepMind admitting the defeat.


After losing three games the self learning human Lee Sedol has finally beaten the DeepMind machine. It is really fascinating to see humans with self learning capabilities can by themselves learn to outsmart machines.
Not knowing the  game of GO, it seems to me it is an ideal board game for computers to play, the basic rules seems simple to turn into algorithms, it is the humongous possible moves that seems to be the challenge. To store data and calculate next move it should just be a matter of capacity. But it is probably trickier than I think. The DeepMind seems to be faster than Lee Sedol who needed one minute slots extra time for the entire end game. I like to see statistics on think time for the players as the game progressed. Actually I like to see stats on everything from these games. And it would be really interesting if DeepMind could explain the reasoning behind every move.
I have written some posts on my thoughts of AI. I do not believe much on the imminent machine takeover, the so called singularity. We have not figured out what human intelligence is, we have come a long way but we are nowhere near of a full description of our intelligence. It may not be necessary to understand our own brain to create a smarter machine brain, but I think it would help if we know ourselves first. The singularity is still 20 years in future, and I predict it will be so for a while.
Nevertheless it is not a small feat for a machine playing GO 3-1 against human. Now I like to see a machine take on something considerable harder, beat human in a game of StarCraft.

Update:
Oh fuck! DeepMind's AlphaGo program won the last match so 4-1 to the machine against the self learning human brain. I had hoped for a second human victory. Still AI is in it's infancy machines are still far from human intellectual capacity. GO is just a deterministic board game, with simple rules. It is the multitude of possible moves that makes it hard for human. And that also makes it hard for human to study and learn from AlphaGo how to beat the program. Which by the way is an interesting idea, self learning humans learn by study machines how to outsmart them. I still put my two cents on humans.

2016-03-07

More parallel processes?

Some time ago I wrote a post about how we improved performance of a job a lot by parallelize the process and introduce a covering index. With the covering index and 10 parallel processes we cut down execution time from +10 hours to a few minutes. The job uses map and reduce design pattern, where 8000 top nodes with some 400000 lower level nodes are assembled into three structures.


Last weekend I decided to see how far I could take the parallelizing within the present infrastructure. How much could we improve the performance by increase the number of threads? Measure parallel processes in a live production environment is quite complex there are many variables to take into account like system load, caches etc. I decided to run my tests during low traffic hours many times with warm caches and buffers,  then take the average from many test runs. The figures were consistent with only small variations so I think result is relevant. As you see in the graph I started with 5 parallel processes and ended with 100 processes. The job scales reasonable well down to about 15 processes, while more than 80 process do not improve the performance much.



Running this job with 10 parallel processes gives a total execution time well below 3 minutes which is much lower than what is required. There is no need to add more parallel processes. But it’s nice to know we can go further.