What is so special about Silicon Valley for technology minded people and why there is so much buzz on the internet about it?
Silicon Valley is unique in that in one small area there is a huge concentration of high tech industry and startup companies. All the facilities are there for anyone wanting to enter the field. There is lots of talent in programming, product design, HR and so on already there, so it's easy to recruit staff, there are plenty of investment "angels" who either live there or visit the place regularly - handy both for investment purposes and for support and advice, and of course the infrastructure is there. In short, it's an entire ecosystem for the high tech industry.
It's not the only place with such characteristics. There are similar places (though much smaller) near Oxford in England, and "Silicon Glen" in Scotland, and Israel is pretty much a Silicon Valley in its own right. Bungalore in India is also a centre for high tech, but "the original" is still the largest by far, and with the highest concentration of brains money and facilities, which is why it is rightfully so well known.
Another important point is that with Stanford University next door there is no shortage on new talents able to grow within your company.
This also goes the other way around ~ there are plenty of opportunities to find work within the same field without having to geographically relocate or take long commutes. One can potentially find their next job just across the street from their old one.
It is something similar to Bangalore in India as it is to the USA. Probably, now you and I being Indians, the similarity will be easier to understand. Just probably Silicon Valley will be at-least 100 times bigger in scale than Bangalore.
There is still a buzz about Silicon Valley? I thought people were over that 10 years ago when everyone realized most electronics come from Asia. Jokes aside, I suggest you read the Wiki article about "Silicon Valley"