Python ollama的搭建与使用流程分析

import ollama
response = ollama.chat(model='llama2', messages=[
  {
    'role': 'user',
    'content': '解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司，Bikky收就行，电话号码13566778899。我的电话是18988998899，上海杨浦区。',
  },
])
print(response['message']['content'])
经过分析，可以知道：
* 收件人地点：上海市，特别是杨浦区
* 公司名称：ABC公司
* 收件人姓名：Bikky
* 收件人电话号码：13566778899
* 你的电话号码：18988998899，也在上海市，但不同区。

mac m1 pro: 平均8秒
[3.5188119411468506,
 5.56521201133728,
 7.0565900802612305,
 11.417732238769531,
 7.563968896865845,
 9.987686157226562,
 6.56359601020813,
 6.939379930496216,
 9.785239219665527,
 11.655837059020996]
In [10]: np.mean(time_list)
Out[10]: 8.005405354499818
2080Ti：平均2.9秒
[2.193615674972534,
 2.8509912490844727,
 2.972665786743164,
 2.2655117511749268,
 3.038464069366455,
 4.976086378097534,
 3.4014697074890137,
 2.209334373474121,
 2.832230567932129,
 2.2567901611328125]
np.mean(time_list)
2.8997159719467165

curl -fsSL https://ollama.com/install.sh | sh

ollama serve

pip3 install ollama -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain_community -i https://mirrors.aliyun.com/pypi/simple/
pip3 install beautifulsoup4 -i https://mirrors.aliyun.com/pypi/simple/
pip3 install faiss-cpu -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain_text_splitters -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain -i https://mirrors.aliyun.com/pypi/simple/

docker pull ollama/ollama

docker run -d \
 --name=ollama01 \
 -v /etc/localtime:/etc/localtime  \
 -v /etc/timezone:/etc/timezone\
 -v /etc/hostname:/etc/hostname \
 -v /data:/root \
 -e "LANG=C.UTF-8"\
 -p 11434:11434\
 -w /workspace \
 --gpus all \
 ollama/ollama

└─ $ docker run  -it --rm --gpus=all registry.cn-hangzhou.aliyuncs.com/andy08008/pytorch_jupyter:v8 bash
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

systemctl restart docker

import ollama
response = ollama.chat(model='llama2', messages=[
  {
    'role': 'user',
    'content': '解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司，Bikky收就行，电话号码13566778899。我的电话是18988998899，上海杨浦区。',
  },
])
print(response['message']['content'])
 Based on the information provided, here is a breakdown of the recipient's details:
* Recipient's location: 上海国金中心 (Shanghai Gold Center) in the 33F floor.
* Company name: ABC公司 (ABC Company).
* Recipient's name: Bikky收就行 (Bikky receives).
* Recipient's phone number: 13566778899.
* Your phone number: 18988998899, located in 上海杨浦区 (Shanghai Yangpu District).
Please note that the above information is based on the text you provided and may not be accurate or up-to-date.

import ollama
response = ollama.chat(model='llama2:13b', messages=[
  {
    'role': 'user',
    'content': '解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司，Bikky收就行，电话号码13566778899。我的电话是18988998899，上海杨浦区。',
  },
])
print(response['message']['content'])
OK! Here's the analysis of the information you provided:
1. Receiver's location: Shanghai, China (based on the address "上海国金中心中心33F")
2. Company name: ABC Company (based on the address "上海国金中心中心33F")
3. Receiver's name: Bikky (based on the name "Bikky收")
4. Receiver's contact information: Phone number 13566778899 (based on the phone number in the address)
5. Sender's location: Shanghai Yangpu District (based on the phone number 18988998899)
6. Sender's name: Not provided
I hope this helps! Let me know if you have any other questions.

from langchain_community.llms import Ollama
llm = Ollama(model="llama2")
llm.invoke("how can langsmith help with testing?")
Langsmith is a tool that can be used to test and validate the correctness of language models. Here are some ways in which Langsmith can help with testing:
1. **Text generation**: Langsmith can be used to generate text samples that can be used to test the language model's ability to produce coherent and contextually relevant text. By comparing the generated text to a reference sample, Langsmith can evaluate the model's performance in terms of fluency, coherence, and relevance.
2. **Text completion**: Langsmith can be used to test the language model's ability to complete partial sentences or phrases. By providing a starting point for the model and evaluating its output, Langsmith can assess the model's ability to capture the context and produce coherent text.
3. **Sentiment analysis**: Langsmith can be used to test the language model's ability to classify text as positive, negative, or neutral. By providing a dataset of labeled text samples and evaluating the model's performance, Langsmith can assess its ability to accurately classify sentiment.
4. **Named entity recognition**: Langsmith can be used to test the language model's ability to identify named entities in text, such as people, organizations, and locations. By providing a dataset of labeled text samples and evaluating the model's performance, Langsmith can assess its ability to accurately identify named entities.
5. **Question answering**: Langsmith can be used to test the language model's ability to answer questions based on a given context or input. By providing a dataset of labeled question-answer pairs and evaluating the model's performance, Langsmith can assess its ability to accurately answer questions.
6. **Dialogue generation**: Langsmith can be used to test the language model's ability to engage in coherent and contextually relevant dialogue. By providing a dataset of labeled dialogue samples and evaluating the model's performance, Langsmith can assess its ability to produce natural-sounding dialogue.
7. **Multi-task evaluation**: Langsmith can be used to test the language model's ability to perform multiple tasks simultaneously, such as language translation, sentiment analysis, and named entity recognition. By providing a dataset of labeled text samples and evaluating the model's performance across multiple tasks, Langsmith can assess its ability to handle multi-tasking.
By using Langsmith to test these aspects of language models, developers can gain a better understanding of their strengths and weaknesses, and make informed decisions about how to improve them.

from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are world class technical documentation writer."),
    ("user", "{input}")
])

chain = prompt | llm 
print(chain.invoke({"input": "how can langsmith help with testing?"}))
As a world-class technical documentation writer, I must say that Langsmith is an excellent tool for testing! Here are some ways in which Langsmith can help with testing:
1. Automated Testing: Langsmith provides automated testing capabilities, allowing you to create and run tests without any manual intervention. This saves time and effort, while also ensuring consistency and accuracy in your test results.
2. Customizable Tests: With Langsmith, you can create customizable tests tailored to your specific needs. You can define the test cases, question types, and evaluation criteria to suit your requirements.
3. Integration with Existing Tools: Langsmith integrates seamlessly with popular testing tools such as JIRA, TestRail, and PractiTest. This enables you to create, run, and manage tests within a single platform, streamlining your testing process.
4. Collaborative Testing: Langsmith supports collaborative testing, allowing multiple team members to work together on a test project. This fosters collaboration and coordination among team members, ensuring that everyone is on the same page and working towards the same goal.
5. Real-time Feedback: Langsmith provides real-time feedback on test results, enabling you to identify areas of improvement immediately. This helps you to rectify errors early on and ensure that your tests are accurate and reliable.
6. Test Case Management: Langsmith offers a comprehensive test case management system, which enables you to organize, track, and maintain your test cases. This helps you to keep your tests organized, up-to-date, and easily accessible.
7. Reporting and Analytics: Langsmith provides detailed reporting and analytics on test results, allowing you to identify trends, strengths, and weaknesses in your testing process. This enables you to optimize your testing strategy and improve overall quality.
8. Integration with Agile Methodologies: Langsmith is designed to work seamlessly with agile methodologies such as Scrum and Kanban. This ensures that your testing activities are aligned with the rest of your development process, enabling you to deliver high-quality software quickly and efficiently.
9. Customizable Workflows: Langsmith allows you to create customizable workflows tailored to your specific needs. This enables you to streamline your testing process and ensure that it aligns with your project's unique requirements.
10. User-Friendly Interface: Langsmith boasts a user-friendly interface, making it easy for team members to use and navigate the platform. This reduces the learning curve and ensures that everyone can use the platform effectively.
In summary, Langsmith is an excellent tool for testing as it provides automated testing capabilities, customizable tests, integration with existing tools, collaborative testing, real-time feedback, test case management, reporting and analytics, integration with agile methodologies, customizable workflows, and a user-friendly interface. These features work together to create a comprehensive and efficient testing platform that can help you deliver high-quality software quickly and efficiently.

from langchain_core.output_parsers import StrOutputParser

output_parser = StrOutputParser()
chain = prompt | llm | output_parser
print(chain.invoke({"input": "how can langsmith help with testing?"}))

As a world-class technical documentation writer, I can help with testing in several ways:

1. Content Creation: Langsmith can assist in creating comprehensive and accurate content for your software or system, including user manuals, technical guides, and release notes. This content can be used to test the system's functionality and usability, ensuring that it meets the requirements and expectations of users.
2. Collaborative Testing: Langsmith can collaborate with your testing team to create test cases and scenarios based on the documentation created. This can help ensure that all aspects of the system are thoroughly tested and that any issues or bugs are identified early in the development process.
3. Automated Testing: By using natural language processing (NLP) and machine learning (ML) algorithms, Langsmith can assist in automating testing processes. For example, Langsmith can be used to generate test cases based on the documentation, or to analyze test results and identify areas for improvement.
4. Test Data Generation: Langsmith can help generate test data that is tailored to your system's requirements. This can include generating sample inputs, outputs, and edge cases that can be used to test the system's functionality.
5. Defect Reporting: Langsmith can assist in identifying and reporting defects or issues found during testing. By analyzing the documentation and test results, Langsmith can generate detailed reports of defects and suggest fixes or improvements.
6. Test Planning: Langsmith can help plan and prioritize testing efforts by analyzing the system's requirements and identifying critical areas that need to be tested. This can help ensure that the most important features and functionality are thoroughly tested, and that testing resources are allocated effectively.
7. Test Execution: Langsmith can assist in executing tests by generating test scripts based on the documentation and test cases created. This can help ensure that all aspects of the system are tested and that any issues or bugs are identified early in the development process.
8. Test Data Management: Langsmith can help manage test data by generating sample inputs, outputs, and edge cases that can be used to test the system's functionality. This can help ensure that test data is accurate, up-to-date, and relevant to the system being tested.
9. Performance Tuning: By analyzing the documentation and testing results, Langsmith can assist in identifying performance issues and suggest optimizations or improvements.
10. Security Testing: Langsmith can help identify security vulnerabilities in the system by analyzing the documentation and test results. This can help ensure that the system is secure and meets the necessary security requirements.

In summary, Langsmith can assist in various testing activities, including content creation, collaborative testing, automated testing, test data generation, defect reporting, test planning, test execution, test data management, performance tuning, and security testing. By leveraging natural language processing and machine learning algorithms, Langsmith can help improve the efficiency and effectiveness of your testing efforts.

from langchain_community.document_loaders import WebBaseLoader
loader = WebBaseLoader("https://docs.smith.langchain.com/user_guide")
docs = loader.load()

from langchain_community.embeddings import OllamaEmbeddings
embeddings = OllamaEmbeddings()
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplitter
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(docs)
vector = FAISS.from_documents(documents, embeddings)

from langchain.chains.combine_documents import create_stuff_documents_chain
prompt = ChatPromptTemplate.from_template("""Answer the following question based only on the provided context:
<context>
{context}
</context>
Question: {input}""")
document_chain = create_stuff_documents_chain(llm, prompt)

from langchain_core.documents import Document
document_chain.invoke({
    "input": "how can langsmith help with testing?",
    "context": [Document(page_content="langsmith can let you visualize test results")]
})
Based on the provided context, Langsmith can help with testing by providing a way to visualize test results. This means that Langsmith can assist in the process of testing software or applications by allowing users to view and analyze the results of those tests in a visual format, such as charts, graphs, or other visualizations.

from langchain.chains import create_retrieval_chain
retriever = vector.as_retriever()
retrieval_chain = create_retrieval_chain(retriever, document_chain)
response = retrieval_chain.invoke({"input": "how can langsmith help with testing?"})
print(response["answer"])
Based on the provided context, LangSmith can help with testing in several ways:
1. Prototyping: LangSmith allows for quick experimentation between prompts, model types, and retrieval strategy, making it easier to understand how the model performs and debug where it is failing.
2. Initial Test Set: LangSmith enables developers to create datasets of inputs and reference outputs, which can be used to run tests on their LLM applications. This helps in identifying regressions with respect to initial test cases.
3. Custom Evaluations: LangSmith provides the ability to run custom evaluations (both LLM and heuristic based) to score test results, allowing developers to assess the performance of their LLM applications more comprehensively.
4. Comparison View: The comparison view in LangSmith enables users to see the results for different configurations side-by-side, helping diagnose regressions in test scores across multiple revisions of the application.
5. Playground Environment: LangSmith provides a playground environment for rapid iteration and experimentation, allowing users to quickly test out different prompts and models without having to run each one individually.