Play Python - Assignment 1

Let's take a look at a versatile and high-level programming language, Python. Python is designed to be a highly readable language. It is designed to have an uncluttered visual layout, frequently using English keywords. Python has a smaller number of syntactic exceptions and special cases than most of the other languages. The beauty of Python acts as it's disadvantage too - The indentation. If we do not indent our code properly, it will throw errors. Let's jump into the basic programs of Python and check how it works.

For basic python tutorials, please visit open tech school. Even in this blog I have used problems from the same source.

The Problem Statement: Suppose you're a greengrocer, and you run a survey to see what radish varieties your customers prefer the most. You have your assistant type up the survey results into a text file on your computer, so you have 300 lines of survey data in the file radishsurvey.txt. Each line consists of a name, a hyphen, then a radish variety.

Dataset

We have to find out - 

• Which radish variety got the most votes (most popular)?
• What are the least popular radish varieties (got the least votes)?
• Did anyone vote twice?

Solution:

Code

 At first we created a dictionary to know the varieties of all the radishes and count their vote in a single pass.

Next , read the data from external file( .txt in this case).

Then we split each lines in two parts- names (the persons to voted) and vote(the name of the radishes) by splitting them across the "-".

To calculate the count of the radishes, we had to further process the data to remove any extra spaces and to avoid the repetition of a single variety of radish to to different cases. So, we converted the entries into upper case and marched ahead.

The next step is pretty simple, where we had to count the votes for each variety of radishes. If the variety is present in the dictionary, then +1 is added to the same, if not, a new entry is made and it's count starts from 1.

Output

The last step is to print the vote count of  the different varieties of radishes. I used pprint() instead of print() because it increases the readability.

Note: The output is in alphabetical order - One of the benefits of pprint().

The "Champion" got the highest votes: 76.

We can also get the solution of the second problem from the output.

The "Red King" variety has got the least votes.

Now we need to find if anyone voted more than once or not.

Code

This one is much simpler than the above problem.


Here, we processed the data to eliminate any extra spaces.




Then, used an if statement to check if the name of the voter is already in the "voted" dictionary, the name will be printed as already voted.










This is a small program to give you an idea how Python works, it's syntax and indentation. The next blog will be again on python and I will try to demonstrate how to use charts (at basic level).

Dating R

What is R?

The 18th letter of alphabet??? Of course it is!! But I am writing about the other R. One of the MVP of Analytics.

R is a programming language and software environment for statistical computing and graphics. The R language is widely used among statisticians and data miners for developing statistical software and data analysis.

(source: Wikipedia) 

 First Hands on Experience with R

Few days back  another noob (me) started to learn R. Thanks to "Google" and "TryR" for providing basic ideas about the commands, functions, syntax and all. 

So, I earned all the badges for R on http://tryr.codeschool.com.

Later, one of my professors suggested to go through https://statsguys.wordpress.com/2014/01/03/first-post/ for the taste of a typical data analytics problem used for prediction for a knowledge program available on KAGGLE.

The problem is to develop a binary classification model to predict which passengers survived the sinking Titanic. In the problem, at first we worked to build an algorithm based on the parameters set in the training dataset then we applied the same algorithm on the test data set to get predict the survivors. 

Lessons Learned

The code is already provided in the above mentioned link, so we didn't have to pull our hairs out or scratch our head. As already mentioned, I am new to this field and I didn't understand many functions used in the code but I was able to understand their approach to the problem. Thanks to STATGUYS again for explaining each steps in detail.

The first step to approach any problem is data exploration. It gives you an idea about the data you are dealing with.

The next and the most critical part as per my opinion is cleaning the data. Implementing this process, we can remove the unwanted variables from my dataset and keep it simple. We can also take care of any missing values which generally create huge problems in results or we encounter horrendous errors. We can make inferences on the missing values based on the other variables related to that missing value.

The third part is training the model we have created. We have a test data set. Using that one, we can train the model to provide optimized and desired output. I am still in dark about the training process (may be using neural network or any other process). Surely, I will dig deep into it and and find out the secret recipe. After the training is done, we can implement the model for the test data set and get the output.

Here as you can see, I made my first submission on KAGGLE. I had to submit the excel sheet that was generated as the output for the test data set.

Anyone who is new to R or want to start R, I would suggest all of you to visit all the above links. It will help you a lot.  Others, wish me luck, so that I can do better and submit my own work on KAGGLE soon.  

Thank You.

Neural... Getting Smarter!!!

 What is Neural Network?

What did we use to know when we were child? Nothing, right? As we grow, the learning process also starts. What is the learning process? We were taught and trained and then we learned.

Neural network is an interconnected group of nodes (similar to the neurons in brain) which is capable of making its own decisions and judgements.

Similar to human brains, we have have to train the neural network and make it learn. For example, if my inputs are 2,4,6, then the output must be 4,8,12. So, during the training process, once we develop the automatic learning mechanism, the neural network practices and starts learning on its own. Next time if the input is 199, it will automatically return 398 as the output.

My First Experience

I came to know about neural network when I designed a Controller to stabilize an Inverted Pendulum. In the project we used neural network to build a self-learning control system, where we had to train the neural network with huge amount of data- sets.

https://www.youtube.com/watch?v=r_viGB6-_MQ-  from this video, one can get the idea about how the inverted pendulum works.

Is Neural Network Getting Smarter?

The neural network has come a long way. It started as a learning machine which is capable of making its own call. Now, it has made its mark in the domain of Analytics. It is used in Predictive Analysis where it processes data from the past and present to estimate future values.

Is neural network really getting starter? We will get to know more about it in the upcoming blogs. Till then visit the below links for more interesting topics-

http://datasceptre.blogspot.in/
datadosage.blogspot.in
http://analyticsyatra.blogspot.in/
http://anuragbarman.blogspot.in

http://praxis.ac.in/programs/business-analytics/