Unlike traditional software, there is no one-size-fits-all solution to ensure security and privacy in large language models (LLMs) applications. From prompt injection to agent misuse, passing between a more classic Model denial of service (DoS) attack, these security issues undermine the achievement of responsible and secure AI systems. While established security practices like input validators, output validators, access control, and minimization are essential components, the dynamic nature of LLMs introduces unique form of complexity. One such challenge is the potential for LLMs to be exploited in the creation of deep fakes, altering videos and audio recordings to falsely depict individuals engaging in actions they never took. When it comes to privacy, several data protection principlesre require a comprehensive reassessment as developers need to navigate a complex terrain of legal and ethical challenges. In this talk, we will explore strategies to ensure the security and privacy of data when using LLMs and discuss best practices for managing data effectively.