Artificial Intelligence and Machine Learning tools in healthcare can help to improve patient outcomes, reduce costs and improve the workplace experience of healthcare practitioners. AI research in healthcare is accelerating rapidly, with multiple applications being demonstrated and integrated with the healthcare ecosystem. However, still there are challenges interms of real time implementation, setting standards for clinical evaluation, defining the performance metrics etc., Also we should be careful about dataset shift, unintended discriminatory bias and generalisation to new populations on health outcomes. When our human biases are amplified by Technology, it is a matter of life and death. So adequate care should be taken while implementing AI Solutions for Healthcare Domain.