How to buy GPU for machine learning?

Inquiring minds want to know, how does one navigate the complex landscape of purchasing a GPU specifically for machine learning purposes? The quest for optimal performance and cost-efficiency is paramount in this field, and with the rapidly evolving technology, it's crucial to make an informed decision. What factors should be considered in selecting a GPU? How do we assess compatibility with our current setup? What are the key performance indicators we should look for? Additionally, are there any hidden costs or pitfalls we should be aware of? Navigating this terrain demands a keen eye and a thorough understanding of the nuances involved.