First of all, I would recommend watching "Linus Torvalds on Git", a Google Tech Talk by the creator of git. It's an hour long, but it's a wonderful introduction to the philosophy behind distributed version control, especially git. It's not a how-to, and it's not even particularly technical. It's actually mostly about project management, and it's quite entertaining.
The core principle to understand is that everyone has their own copy of the repository. People collaborate by sending and receiving sets of changes. Grabbing a set of changes from somone and incorporating them into your repository is called "pulling". Sending a set of changes from your repository to another one is called "pushing".
Does this kind of project need to be a Private Repository?
On a free GitHub account, your project's source code and its history are public, viewable by anyone. Anyone can click the "Fork" button and make their own copy of your GitHub project. Their copy will be clearly labeled as a fork of your original. (For example, take a look at the upper left corner of this page, it says "daxelrod/pod-pseudopod", and then under it, "forked from allisonrandal/pod-pseudopod".)
By default, you are the only one who can submit changes to your copy of the project. If someone else has made an improvement to their copy of the project, they can send you a Pull Request; that is, a request that you incorporate their improvement into your copy of the project. It is entirely your decision whether you accept a pull request.
As the admin of your copy of the repository, you can add collaborators, if you wish. A collaborator is a specific person who you trust to submit changes directly to your copy of the project. This is entirely optional, and most GitHub projects do not have collaborators; they simply accept pull requests.
A paid GitHub account gives you the ability to create private repositories. A private repository is only viewable and forkable by specific people you have chosen to be private collaborators.
What about readme file does it have to be really ReadMe.MD file.
It's enough to commit a plain text file named README. If you want to get fancy with formatting, you can make a README.md file and use Markdown in it; the same syntax that you use to format Stack Overflow questions.
Google didn't give me Full answers to this there are bits thrown all over the place with little info in them. Provide me a source
Allow me to editorialize for a moment. Git is, at heart, a command-line program, and so most of the reference you'll see involves interacting with it from the command line. I personally recommend you learn git using the command line. That way, you'll understand what's going on, and you'll be able to use the vast majority of help and documentation. I only recommend using a GUI once you know how to use Git from the command line.
If you do want to learn command-line git, Pro Git is a fantastic resource. It's available either as a book, or free online. Honestly, even if you're not learning command-line Git, it may be a good introduction to the concepts and workflows.
If you insist on a GUI, however, take a look at An Illustrated Guide to Git on Windows, which uses git-gui. If you do this please don't be scared off by documentation or tutorials which use the command line! Their principles still apply, except that you're clicking on commands rather than typing them. (And yes, I know that's an oversimplification.)
Your initial GitHub workflow to create a project, commit your first change, and push it, is detailed here - again, with command line instructions. The basic idea is that you have a repository on GitHub, and a clone of it on your local hard drive. You commit changes to your local repository, then you push them to the one on GitHub when you're ready to share them.