Hosting and Comparing Phi-3 mini and Mistral 7B Models for simulating sales scenarios
In this tutorial, we’ll demonstrate how to host two large language models, Phi-3 and Mistral, to talk to each other using Covalent Cloud.
We’ll start by deploying the models on Covalent Cloud, then set up a function to simulate the interaction, and finally, run the simulation to observe how the models perform in their respective roles. This approach not only highlights the ease of deploying and managing models on Covalent Cloud but also demonstrates a practical application of these models in an enterprise context.
Local Environment
Let's start by importing the Covalent Cloud SDK.
import covalent_cloud as cc
Save your API key if you have not done so already.
# cc.save_api_key("you-api-key")
Cloud Environment
Here, we create a cloud environment with the required packages. We'll use vllm as a shortcut to run performant models on GPUs. To learn more about creating runtime environments in Covalent Cloud, see here.
cc.create_env("vllm serve", pip=["vllm==0.5.1", "torch==2.3.0"], wait=True)
Environment Already Exists.
Define a Set of Compute Resources
Let's set up a Cloud Executor which uses 1 A100 GPU (for each deployment). See here for more information on specifying compute resources.
executor = cc.CloudExecutor(
env="vllm serve",
num_cpus=10,
num_gpus=1,
memory="100GB",
gpu_type=cc.cloud_executor.GPU_TYPE.A100,
time_limit="1 hours",
)
Creating Services
On to the actual deployment code! Use the @cc.service decorator and the endpoint decorator to define services. See here for a more detailed explanation.
@cc.service(executor=executor,name="llm models",auth=False) # Note one can also define compute shares etc.. which will define the
def vllm_serve(model):
from vllm import LLM
llm = LLM(model=model, trust_remote_code=True, enforce_eager=True)
return {"llm": llm}
@vllm_serve.endpoint("/generate")
def generate(llm, prompt, num_tokens=1500,temperature=0.7, top_p=0.8):
"""Accepts either a single prompt or a list of prompts and returns the generated text for each prompt."""
from vllm import SamplingParams
sampling_params = SamplingParams(temperature=temperature, top_p=top_p, max_tokens=num_tokens)
is_single_prompt = isinstance(prompt, str)
prompts = [prompt] if is_single_prompt else prompt
outputs = llm.generate(prompts, sampling_params)
generated_texts = [output.outputs[0].text for output in outputs]
return generated_texts[0] if is_single_prompt else generated_texts
Note, also, that you can deploy the above models directly (i.e. outside a Covalent workflow) by running following
# cc.deploy(vllm_serve)(model="microsoft/Phi-3-mini-4k-instruct")
Let's make things more interesting by deploying multiple models at the same by using Covalent workflows.
Deploying services from workflows allows for complex orchestration with other services and high-compute tasks. (See here for more information.) In this case, we'll create a workflow that simply calls the @cc.service-decorated functions inside the workflow body.
import covalent as ct
wf_executor = cc.CloudExecutor(env="vllm serve",num_cpus=5,memory="2GB")
@ct.lattice(executor=wf_executor,workflow_executor=wf_executor)
def compare_models(models):
services = [vllm_serve(model=model) for model in models]
return services
runid = cc.dispatch(compare_models)(models=["microsoft/Phi-3-mini-4k-instruct", "unsloth/mistral-7b-instruct-v0.3"])
The returned object (i.e. the result value) will be a Python client that grants access to every endpoint in a given service.
run_result=cc.get_result(runid, wait=True)
run_result.result.load()
deployed_models = run_result.result.value
Let's quickly test the deployment by making an inference call to the model. We're using a Python client here to query the hosted API, but generic HTTP requests will work as well, using cURL or Postman for example.
for model in deployed_models:
print(model)
╭────────────────────────────── Deployment Information ──────────────────────────────╮
│ Name Microsoft-Phi-3-mini │
│ Description Add a docstring to your service function to populate this section. │
│ Function ID 6650eb02f7d37dbf2a468bbe │
│ Address https://fn.prod.covalent.xyz/16650eb02f7d37dbf2a468bbe │
│ Status ACTIVE │
│ Tags │
│ Auth Enabled No │
╰────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ POST /generate │
│ Streaming No │
│ Description Accepts either a single prompt or a list of prompts and returns the generated text for each │
│ prompt. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────── Deployment Information ──────────────────────────────╮
│ Name Mistral-7b-instruct │
│ Description Add a docstring to your service function to populate this section. │
│ Function ID 6650eb08f7d37dbf2a468bc0 │
│ Address https://fn.prod.covalent.xyz/16650eb08f7d37dbf2a468bc0 │
│ Status ACTIVE │
│ Tags │
│ Auth Enabled No │
╰────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ POST /generate │
│ Streaming No │
│ Description Accepts either a single prompt or a list of prompts and returns the generated text for each │
│ prompt. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Generating Responses
[model.generate(prompt="Hi ! ",num_tokens=20) for model in deployed_models]
["😊 I'm glad I could assist you. Remember, if you have any more",
" I'm a newbie here, and I'm trying to figure out how to add a"]`
To illustrate an example, let's use the deployed models to simulate a scenario where one model acts as a "sales representative" and the other as a "demanding client".
This simulation can provide valuable insights into handling challenging sales calls, perhaps for training of sales reps in realistic scenarios. By examining how the models interact, one can gather information on effective strategies for client interaction. This approach can also be useful for refining sales pitches, understanding client objections, and improving customer interactions overall.
The following code sets up and runs this simulation.
from IPython.display import display, HTML, clear_output
class Player:
def __init__(self, llm, role, emoji, prompt_template, name):
self.llm = llm
self.role = role
self.emoji = emoji
self.prompt_template = prompt_template
self.name = name
self.chat_history = []
def generate_response(self, prompt):
if self.chat_history:
prompt = "\n".join(self.chat_history) + "\n" + prompt
prompt = self.prompt_template.format(prompt)
response = self.llm.generate(prompt=prompt, num_tokens=500)
self.chat_history.append(f"{self.role}: {response}")
return response
class SalesRep(Player):
def pitch_product(self, product_description):
prompt = f"As a sales representative, your goal is to sell the following product:\n\n{product_description}\n\nProvide a persuasive pitch that highlights the key benefits and unique features of the product. Make it conversational as if you are talking to a potential client. Make it short."
return self.generate_response(prompt)
def respond_to_objection(self, client_objection):
prompt = f"As a sales representative, respond to the following objection from a potential client:\n\n{client_objection}\n\nProvide a clear and convincing response that addresses the client's concerns. Make it conversational as if you are talking to a potential client. Make it short, yet informative."
return self.generate_response(prompt)
class ToughClient(Player):
def ask_difficult_question(self, product_description):
prompt = f"As a tough client, your goal is to scrutinize the following product:\n\n{product_description}\n\nAsk a challenging question that addresses potential weaknesses or concerns about the product. Make it conversational as if you are talking to a sales representative. Make it short."
return self.generate_response(prompt)
def simulate_sales_interaction(sales_llm, client_llm, product_description, num_iterations=5):
sales_rep = SalesRep(sales_llm, "SalesRep", "💼 SalesRep:", sales_llm.prompt_template, sales_llm.name)
tough_client = ToughClient(client_llm, "Client", "👤 Client:", client_llm.prompt_template, client_llm.name)
def display_conversation(chat_history):
html_content = """
<style>
.chat-box {
width: 80%;
max-width: 800px;
margin: auto;
font-family: Arial, sans-serif;
color: #f1f1f1;
}
.chat {
border: 1px solid #ccc;
padding: 10px;
margin: 10px;
border-radius: 10px;
}
.chat-sales {
background-color: #2B2F3A;
text-align: left;
color: #61dafb;
}
.chat-client {
background-color: #1E2229;
text-align: right;
color: #f1f1f1;
}
.chat-box-title {
text-align: center;
font-size: 1.5em;
margin-bottom: 20px;
color: #61dafb;
}
</style>
<div class="chat-box">
<div class="chat-box-title">Sales Pitch and Client Interaction</div>
"""
for role, emoji, name, response in chat_history:
user_class = "chat-sales" if role == "SalesRep" else "chat-client"
html_content += f"""
<div class="chat {user_class}">
<p><strong>{emoji} {name}:</strong></p>
<p>{response}</p>
</div>
"""
html_content += "</div>"
clear_output(wait=True)
display(HTML(html_content))
# Initial product pitch by the sales representative
sales_pitch = sales_rep.pitch_product(product_description)
chat_history = [(sales_rep.role, sales_rep.emoji, sales_rep.name, sales_pitch)]
display_conversation(chat_history)
for _ in range(num_iterations):
# Client asks a difficult question
client_question = tough_client.ask_difficult_question(product_description)
chat_history.append((tough_client.role, tough_client.emoji, tough_client.name, client_question))
display_conversation(chat_history)
# Sales representative responds to the objection
sales_response = sales_rep.respond_to_objection(client_question)
chat_history.append((sales_rep.role, sales_rep.emoji, sales_rep.name, sales_response))
display_conversation(chat_history)
model_names = ["Microsoft-Phi-3-mini", "Mistral-7b-instruct"]
prompt_templates = ["<|user|>\n{} <|end|>\n<|assistant|>", "[INST] {} [/INST]"]
for model, name, prompt_template in zip(deployed_models, model_names, prompt_templates):
model.name = name
model.prompt_template = prompt_template
llm1, llm2 = deployed_models
# Example usage
product_description = "AI-powered CRM system that offers advanced data analytics, seamless integration with \
existing tools, and a user-friendly interface designed to boost sales productivity and customer satisfaction."
Phi-3 as Sales Representative
simulate_sales_interaction(sales_llm=llm1, client_llm=llm2, product_description=product_description, num_iterations=5)
Hey there! I'm excited to introduce you to our game-changing AI-powered CRM system, designed to revolutionize your sales process. Imagine having a tool that not only stands your customers' needs but also predicts their future behavior, enabling you to deliver personalized experiences like never before. First off, our system offers advanced data analytics, providing you with deep insights into your customer base and sales trends. This empowers you to make informed decisions and strategize effectively for your business growth. Now, we know you're already using a variety of tools, and our CRM system ensures seamless integration with your existing setup. This means no disruption in your workflow, and everything you need is just a click away. But that's not all! Our CRM system features a user-friendly interface, making it easy for you and your team to navigate and utilize its capabilities. This results in increased productivity, as your team can focus on what they do best - selling! Lastly, our AI-powered system is built to boost customer satisfaction. By leveraging the power of AI, it helps you understand your customers better, tailor your offerings, and ultimately, provide an exceptional experience. In summary, our AI-powered CRM system will transform your sales process, providing you with actionable insights, seamless integration, and an intuitive user experience. Ready to take your sales productivity and customer satisfaction to new heights? Let's talk more about how this innovative solution can benefit your business.
Given the AI-powered nature of the CRM system, I'm curious about how the system handles data privacy and security, especially with the increasing concerns around AI and data breaches. Can you assure me that the system complies with all relevant data protection regulations and employs robust security measures to protect sensitive customer information?
Absolutely, data privacy and security are paramount for us, especially considering the sensitive nature of customer information. Our AI-powered CRM system is built with robust security measures in place to ensure the protection of your data. We strictly adhere to all relevant data protection regulations, such as GDPR and CCPA, and our system is designed to comply with these standards. To further enhance data security, we employ advanced encryption techniques, multi-factor authentication, and strict access controls to safeguard your information. Additionally, our team continuously monitors and updates our security protocols to stay ahead of potential threats. Rest assured, we understand the importance of data privacy and have taken every step to ensure that our AI-powered CRM system maintains the highest level of security and compliance. If you have any specific concerns or questions about our security measures, I'd be more than happy to address them.
Given the sophisticated nature of the AI-powered CRM system, I'm particularly interested in understanding how the system mitigates potential biases in its data analytics, ensuring fair and unbiased treatment of all customers. Could you elaborate on the measures taken to prevent AI algorithms from perpetuating or amplifying existing biases in our data, which could potentially harm our customer relationships and reputation?
Absolutely, I completely understand your concern about potential biases in AI-powered systems. At our company, we've prioritized the development of our AI-powered CRM system with a strong focus on fairness and unbiased data analysis. Firstly, we've designed our system to be transparent and explainable. This means that our AI algorithms can provide clear insights into how they arrived at their conclusions. This transparency helps us identify and address any potential biases that may exist within the data or the algorithms themselves. Secondly, we've implemented a rigorous data cleansing and preprocessing process. By ensuring that our data is accurate, complete, and representative of our diverse customer base, we minimize the risk of biased outcomes. Our data scientists work diligently to identify and eliminate any data inconsistencies or inaccuracies that could potentially skew results. Moreover, we continuously monitor and evaluate the performance of our AI algorithms. By regularly assessing the outcomes and comparing them to our expected results, we can identify any discrepancies and take corrective actions if needed. This ongoing evaluation helps us maintain the fairness and accuracy of our data analytics. Lastly, we're committed to learning and improving. As we gather more data and feedback, we're constantly refining our AI algorithms to reduce biases and ensure unbiased treatment of all customers. We also encourage our customers to provide feedback on the system's performance, so we can address any concerns and make necessary adjustments. I hope this addresses your concerns. Our AI-powered CRM system is designed with fairness and unbiased data analysis in mind, and we're dedicated to ensuring that our customers receive the best possible service and experience. If you have any further questions or need more information, please don't hesitate to ask.
Given the advanced nature of the AI-powered CRM system, I'm concerned about the system's transparency and explainability of its decision-making process, especially when it comes to data analytics. How does the system ensure that its AI algorithms are not only accurate but also understandable, allowing us to trace and correct any errors or unintended biases that may arise?
Thank you for raising this important concern! We completely understand the need for transparency and explainability in AI-powered systems, especially when it comes to decision-making processes. Our AI-powered CRM system is designed with these principles in mind, ensuring that our algorithms are both accurate and understandable. To achieve this, we've implemented a few key features:
- Explainable AI (XAI): Our system incorporates XAI techniques, which allow users to understand how the AI algorithms arrived at specific conclusions. This means you can trace the decision-making process and identify any errors or biases that may have occurred.
- Transparent data processing: Our data cleansing and preprocessing processes are designed to be transparent, so you can see exactly how data is being handled and transformed. This ensures that the data used by the AI algorithms is accurate, representative, and free from biases.
- Regular monitoring and evaluation: We continuously monitor and evaluate the performance of our AI algorithms, comparing the outcomes with expected results. This helps us identify any discrepancies and take corrective actions to maintain accuracy and fairness.
- Ongoing improvement: As we gather more data and feedback, we're constantly refining our AI algorithms to reduce biases and improve accuracy. We also encourage our customers to provide feedback on the system's performance, so we can address any concerns and make necessary adjustments. We believe that these features, combined with our commitment to data privacy and security, make our AI-powered CRM system a reliable and trustworthy solution for your sales process. If you'd like to learn more about how our system can help you achieve your business goals while ensuring transparency and fairness, I'd be happy to provide further details or schedule a demo for you.
Given the advanced capabilities of the AI-powered CRM system, I'm interested in understanding how the system handles situations where the AI's predictions or recommendations may conflict with human judgment or ethical standards. Could you provide examples of safeguards in place to ensure that the system prioritizes ethical decision-making and avoids inappropriate or unethical behavior, while still maintaining its high level of accuracy and efficiency?
Absolutely, I'm glad you brought up this important aspect! Our AI-powered CRM system is designed with ethical considerations at its core. Here are some ways we ensure that the system prioritizes ethical decision-making:
- Human-in-the-loop (HITL): Our system incorporates a human-in-the-loop approach, meaning that critical decisions are reviewed and approved by a human agent. This ensures that the AI's recommendations are aligned with human judgment and ethical standards. The AI acts as an assistant, providing insights and suggestions, but the final decision always rests with the human agent.
- Ethical guidelines: We've established a set of ethical guidelines that govern the AI's behavior and decision-making process. These guidelines are based on industry best practices and are continuously updated to reflect evolving ethical standards. The AI system is programmed to follow these guidelines, ensuring that its actions remain ethical and aligned with our values.
- Transparency and explainability: As mentioned earlier, our system incorporates explainable AI (XAI) techniques, which allow users to understand how the AI arrived at specific conclusions. This transparency helps us identify any potential ethical concerns and address them proactively.
- Regular monitoring and auditing: We continuously monitor and audit the AI system to ensure that it remains ethical and aligned with our values. This includes reviewing the AI's decisions, analyzing data patterns, and assessing the system's overall performance. Let me give you an example to illustrate this. Suppose the AI system predicts that a customer may be more receptive to a specific product offer. However, the human agent, considering the customer's past interactions and preferences, decides that it would be more appropriate to offer a different product. In this case, the human agent's decision takes precedence, and the AI system will adapt its recommendations accordingly. Our goal is to create a balance between leveraging the AI system's powerful capabilities while maintaining ethical decision-making and human oversight. We believe that this approach ens
Client: Given the complexity of AI systems, I'm curious about the system's ability to adapt and learn from its mistakes, ensuring continuous improvement and avoiding the risk of reinforcing errors or biases over time. Can you elaborate on the system's mechanisms for self-learning and error correction, and how it ensures that these improvements don't come at the expense of data privacy or security?
Absolutely! It's great that you're considering the system's continuous improvement and data privacy aspects. Our AI-powered CRM system is designed with self-learning capabilities, which allow it to adapt and improve over time while prioritizing data privacy and security. Here's how it works:
- Data feedback loops: The AI system learns from the outcomes of its recommendations and decisions. By analyzing the results and comparing them to the desired outcomes, the AI can identify and learn from any mistakes, ensuring that it continuously improves its accuracy and effectiveness.
- Human-in-the-loop (HITL): As mentioned earlier, the AI system is not entirely autonomous. Human agents are always involved in the decision-making process, reviewing and approving the AI's recommendations. This ensures that the system's learning process aligns with human judgment and ethical standards, minimizing the risk of reinforcing errors or biases.
- Continuous monitoring and auditing: We have a robust monitoring and auditing system in place to track the AI's learning progress and ensure that it's not reinforcing errors or biases. This involves analyzing data patterns, reviewing the AI's decisions, and assessing its overall performance.
- Data privacy and security: All learning and self-improvement processes take place while strictly adhering to data privacy and security regulations. Our system employs advanced encryption, multi-factor authentication, and strict access controls to protect sensitive data. Additionally, we regularly update our security protocols to address emerging threats and maintain the highest level of data protection. In summary, our AI-powered CRM system is designed to learn and improve continuously, but always under human supervision and with strict data privacy and security measures in place. This ensures that you can trust the system to evolve and become even more effective over time, while keeping your data safe.
Mistral as Sales Representative
simulate_sales_interaction(sales_llm=llm2, client_llm=llm1, product_description=product_description, num_iterations=5)
Hey there! I'm excited to introduce you to our AI-powered CRM system, a game-changer for your sales team. Imagine a tool that not only streamlines your sales process but also amplifies productivity and customer satisfaction. Our system offers advanced data analytics, transforming raw data into actionable insights. It helps you identify trends, predict customer behavior, and make data-driven decisions, all in real-time. But that's not all! It seamlessly integrates with your existing tools, eliminating the need for multiple platforms. This integration saves time, reduces errors, and allows your team to focus on what truly matters - closing deals. The user-friendly interface is designed with your sales team in mind. It's intuitive, easy to navigate, and reduces the learning curve, allowing your team to hit the ground running. In essence, our AI-powered CRM system is not just a tool, but a partner that boosts your sales productivity and enhances customer satisfaction. Let's schedule a demo and see how it can revolutionize your sales process!
How does the AI-powered CRM ensure data privacy and security, especially when integrating with various existing tools? Can it adapt to different industries' regulatory requirements?
Absolutely, I understand your concerns about data privacy and security, especially when integrating with various tools. Our AI-powered CRM takes these matters very seriously. Firstly, we employ state-of-the-art encryption techniques to ensure all data is securely transmitted and stored. We also comply with industry-standard security protocols such as GDPR, CCPA, and others, ensuring we adhere to global data protection regulations. Secondly, our system is designed to adapt to different industries' regulatory requirements. We provide customizable data privacy settings that allow you to control who has access to what data. Moreover, we regularly update our system to stay compliant with evolving regulations. During the demo, I'd be happy to walk you through our security measures and show you how our system can be tailored to your specific industry's needs. Let's schedule that demo and explore how our AI-powered CRM can help you achieve your sales goals while ensuring the highest level of data privacy and security.
How does this AI-powered CRM system guarantee compliance with diverse industry-specific data protection regulations, and what measures are in place to prevent unauthorized data access or breaches when integrating with external tools?
I'm glad you've raised that important point. Our AI-powered CRM system ensures compliance with diverse industry-specific data protection regulations by adapting to each industry's unique requirements. We achieve this through customizable data privacy settings that allow you to control who has access to what data. Regarding data security when integrating with external tools, we employ state-of-the-art encryption techniques to ensure all data is securely transmitted and stored. We also comply with industry-standard security protocols such as GDPR, CCPA, and others. To prevent unauthorized data access or breaches, we regularly update our system to stay compliant with evolving regulations and conduct regular security audits. During the demo, I'd be more than happy to walk you through these measures and show you how our system can be tailored to your specific industry's needs. Let's schedule that demo and explore how our AI-powered CRM can help you achieve your sales goals while ensuring the highest level of data privacy and security.
How does the AI-powered CRM ensure robust data security during integrations, and can it dynamically adjust to various industry-specific regulations like GDPR or HIPAA? This question challenges the product's ability to handle data security during integrations and its adaptability to different regulations, which are crucial concerns for any CRM system dealing with sensitive data.
Absolutely, I understand your concerns about data security, especially during integrations. Our AI-powered CRM employs state-of-the-art encryption techniques to ensure all data is securely transmitted and stored. We also comply with industry-standard security protocols such as GDPR, CCPA, and others, ensuring we adhere to global data protection regulations. Moreover, our system is designed to adapt to different industries' regulatory requirements. We provide customizable data privacy settings that allow you to control who has access to what data. To prevent unauthorized data access or breaches, we regularly update our system to stay compliant with evolving regulations and conduct regular security audits. In terms of industry-specific regulations like GDPR or HIPAA, our system can dynamically adjust to meet these requirements. During the demo, I'd be more than happy to walk you through these measures and show you how our system can be tailored to your specific industry's needs. Let's schedule that demo and explore how our AI-powered CRM can help you achieve your sales goals while ensuring the highest level of data privacy and security.
How does the AI-powered CRM system ensure that data privacy and security measures are consistently updated and maintained, particularly when integrating with new tools or adapting to emerging industry regulations? Can you provide examples of how the system has successfully navigated complex regulatory landscapes?
Absolutely, I understand your concerns about data security, especially during integrations. Our AI-powered CRM employs state-of-the-art encryption techniques to ensure all data is securely transmitted and stored. We also comply with industry-standard security protocols such as GDPR, CCPA, and others, ensuring we adhere to global data protection regulations. Moreover, our system is designed to adapt to different industries' regulatory requirements. We provide customizable data privacy settings that allow you to control who has access to what data. To prevent unauthorized data access or breaches, we regularly update our system to stay compliant with evolving regulations and conduct regular security audits. In terms of industry-specific regulations like GDPR or HIPAA, our system can dynamically adjust to meet these requirements. During the demo, I'd be more than happy to walk you through these measures and show you how our system can be tailored to your specific industry's needs. Let's schedule that demo and explore how our AI-powered CRM can help you achieve your sales goals while ensuring the highest level of data privacy and security.
How does the AI-powered CRM system ensure that data privacy and security measures are consistently updated and maintained, particularly when integrating with new tools or adapting to emerging industry regulations? Can you provide examples of how the system has successfully navigated complex regulatory landscapes?
SalesRep: I'm glad you've raised that question. Our AI-powered CRM system is committed to maintaining the highest level of data privacy and security. To ensure our measures are consistently updated and maintained, we regularly conduct security audits, stay informed about emerging industry regulations, and collaborate with cybersecurity experts. For instance, when integrating with new tools, we employ secure API connections and ensure that the third-party tools we work with also comply with our stringent security standards. In terms of adapting to complex regulatory landscapes, our system has successfully navigated GDPR, CCPA, and HIPAA compliance for various clients in different industries. During the demo, I'd be more than happy to provide examples of how our system has successfully navigated these complex regulatory landscapes and show you how we ensure our data privacy and security measures are consistently updated and maintained. Let's schedule that demo and explore how our AI-powered CRM can help you achieve your sales goals while ensuring the highest level of data privacy and security.
Client: In an era where data breaches are common, how does the AI-powered CRM's security measures stand out, and can you share specific instances where it has successfully mitigated risks associated with new integrations or changing regulations? This question targets the AI-powered CRM's security features, its ability to handle new integrations and evolving regulations, and seeks concrete examples of its effectiveness in these areas. It's concise and conversational, fitting for a discussion with a sales rep.
SalesRep: Great question! Our AI-powered CRM stands out due to our proactive approach to security. We regularly conduct security audits, stay informed about emerging industry regulations, and collaborate with cybersecurity experts to ensure our measures are consistently updated and maintained. For instance, when integrating with new tools, we employ secure API connections and ensure that the third-party tools we work with also comply with our stringent security standards. This approach has helped us successfully navigate complex regulatory landscapes like GDPR, CCPA, and HIPAA for various clients in different industries. One example that comes to mind is a healthcare client we worked with. When HIPAA compliance became mandatory for their industry, our AI-powered CRM was able to dynamically adjust to meet these requirements. This allowed them to continue their operations without interruption, ensuring the privacy and security of their sensitive patient data. During the demo, I'd be more than happy to provide more examples and show you how our system can help you navigate complex regulatory landscapes and mitigate risks associated with new integrations. Let's schedule that demo and explore how our AI-powered CRM can help you achieve your sales goals while ensuring the highest level of data privacy and security.
Clean Up
Both model will auto-terminate after 1 hour, as specified in the executor. To delete them manually, use the client's .teardown() method. For example:
[model.teardown() for model in deployed_models]
['Teardown initiated asynchronously.', 'Teardown initiated asynchronously.']
Conclusion
In this tutorial, we used Covalent Cloud to serve two powerful language models, Phi-3 and Mistral, at the same time.
We then loaded a Python client for each service and simulated interactions between a "sales representative" (one model) and a "demanding customer" (the other).
Try using this tutorial as a framework for your own multi-model deployments! This example is easily be adapted to simulate a different set of interactions, experiment with different models, or to introduce new ones into the mix.
All this and more is possible on the Covalent Cloud platform!