Datascienceworld.Kanen

[ certificate - Building Multimodal Search and RAG]

I have recently completed the "Building Multimodal Search and RAG" course and earned a certificate. The course introduced a comprehensive framework for implementing RAG and recommender systems for movies, leveraging multiple input formats such as text, posters, and video.

A key takeaway was the importance of multimodal models, which can effectively integrate and utilize information across different formats. By applying advanced contrastive learning algorithms, the course demonstrated how the semantic connections could be established between text, images, and audio.

This approach enables embedding models to link and relate similar content, even when it exists in different formats. This course brings a good proxy for me to upgrade and reinforce my searching ability for aucodb and vinagent libraries in the next steps.

Thank Weaviate and DeepLearning.AI co-team for contributing a very useful course to the AI community.



https://learn.deeplearning.ai/accomplishments/b4435292-55ce-413d-bede-fc7d17dfcf0d?usp=sharing

I know many of you feel stressed and overwhelmed when working in Ph.D program Computer Science and Data Science. You often have to go through numerous research papers every day. That is why a Research Agent is essential to help expedite your reading process. Equipped with a set of handy scientific research tools, it allows you to connect to a collection of well-known papers from the arXiv site and carry out the following tasks:

- Read top trending AI-related papers.
- Summarize the content of a paper.
- Compare two or more papers to highlight key differences and insights.
- Conduct a literature review.
- Expand your research scope across multiple domains.

Let’s learn how to initialize the Research Agent using the Vinagent library:
https://datascienceworld-kan.github.io/vinagent/guides/paper_research/

[Observability and Tracing AI Agent in Vinagent Library]

Dear friends,

On behalf of the Vinagent development team, I’m excited to introduce a new feature that makes it easier to monitor, measure processing speed, and evaluate the performance of AI Agents. This feature is now directly integrated into the Vinagent library, eliminating the need to set up connections to external API services like LangSmith, Langfuse, or Phoenix. All logging data is kept secure and private by being stored locally on your logging server.

Integration is simple through a local logging database connection as follows:

import mlflow
from vinagent.mlflow import autolog

# Enable Vinagent autologging
autolog.autolog()

# Optional: Set tracking URI and experiment
mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("Vinagent")

With this feature, you can observe intermediate messages from the agent directly within the Jupyter Notebook UI. It shows token counts, processing time for each message, model names used, and the processing status at each step. This is especially useful for those looking to optimize the design of AI Agents.

- I’ve prepared a tutorial video on YouTube for you to follow along:

https://youtu.be/UgZLhoIgc94?si=gebkIk3iW24IL6Ef

To start using the library, follow these steps:

- Step 1: Install the library

pip install vinagent

- Step 2: Set environment variables in a .env file to use APIs for LLM models and search tools:

TOGETHER_API_KEY="Your Together API key"
TAVILY_API_KEY="Your Tavily API key"

- Step 3: Run the model observability feature:
https://github.com/datascienceworld-kan/vinagent #10-agent-observability

We hope this new feature will be helpful in your AI Agent development journey and inspire young innovators in technology.

Sincerely,
The Vinagent Library Development Team

Vinagent's Observability and Tracing feature Vinagent provides a local server that can be used to visualize the workflow and debug the agent. We leverage mlflow observability to track the agent's progre...

[AucoDB – Building a Dedicated Database for LLMs & AI Agents]

Throughout my journey working with large language models (LLMs) and Agent systems, I’ve constantly asked myself:

1. Why is there still no database that truly understands and effectively supports LLM systems?

2. Why is knowledge processing for documents still so fragmented and cumbersome especially when we're aiming to build an effective knowledge base for LLMs and Agents?

3. The world has so many databases, but none of them are truly comprehensive and capable of becoming the go-to solution for knowledge storage tailored for LLMs and Agents.

And I want to make that dream a reality. I want to build a database that solves the core challenges LLMs and Agents face today. I’ve tried MongoDB, PostgreSQL, FAISS, Neo4j… Each tool has its strengths, but when applied to complex LLM problems, they fall short:

1. Traditional databases can store text, but lack an understanding of graph and tabular knowledge.

2. Vector databases excel at search but struggle with organizing semantically rich and meaningful data.

3. Graph databases are powerful at representing relationships but lack standardized input formats for raw documents.

4. Most importantly: No tool currently integrates all forms of information text, tables, vectors, and graph into a seamless workflow.

That’s why AucoDB is systematically addressing these shortcomings. This is not just a new database. It’s a new vision for how data should be stored and harnessed for LLMs and Agents. Introducing AucoDB 0.1.7, a new version designed specifically for data modeling tailored to LLMs and AI Agents:

1. Smart document-based storage like MongoDB, enhanced with APIs that enable quick interactions (add, delete, update, filter, sort).

2. Hybrid knowledge structuring for Agent Memory & Knowledge Base, which combines elements of Graph and Vector databases to support complex AI workflows.

3. Automatic knowledge graph generation from text - using LLMs for semantic parsing and saving directly into Neo4j.

4. Integration with over 7 popular vector databases: Chroma, FAISS, Milvus, PGVector, Pinecone, Qdrant, Weaviate.

5. Easy-to-use in Python and supports JSON export for further processing anywhere.

If you’re a developer building AI applications, a data engineer organizing knowledge, or a researcher exploring data for LLMs and AI Agents - AucoDB is made for you.

Check out the latest release AucoDB 0.1.7 at: https://github.com/datascienceworld-kan/aucodb

[Building AI Browsers Agents course - DeepLearning.AI]

The fast-growing development of Agentic AI in 2025 has ignited a fierce race among many startups, and big techs have joined in. Think of familiar names like Google, Anthropic, Langchain, CrewAI. DataScienceWorld.Kanen is not out of the race either. We have launched a new agentools library that supports connecting thousands of tools to a unique LLM-powered agent. We leverage LLM to accurately pick up actions to handle a load of difficult tasks. If you are an AI-savvy tech, let's try this package at:
https://github.com/datascienceworld-kan/agentools

Furthermore, I have just finished "Building AI Browsers Agents" course at DeepLearning AI platform:
https://learn.deeplearning.ai/accomplishments/28344ee8-c6cb-4bf8-b20f-fd8d1b8df5f6?usp=sharing

This course expanded my general knowledge of how to design a web browser agent with very practical capabilities like handling very complex multi-stage tasks like searching and summarizing a website, or even subscribing to a course or service. Exceptionally, one standout tool is Agent Q, a useful library that extends the model tree-of-thought. It allows for optimizing the strategy of searching over a multi-branch planning tree.

My thought: It seems to be a great idea to integrate into agentools a new tool like a web browser to make a powerful AI agent.

Khánh Phạm Đình, congratulations on completing Building AI Browser Agents! Khánh Phạm Đình, congratulations on completing Building AI Browser Agents!

Hi everyone,

I have just completed Long-Term Agentic Memory With LangGraph course. Let me share with you what I achieved from this course. This course has given me a new perspective on designing Agentic Memory, emphasizing that it should be thoughtfully segmented into:
- Semantic: for fact things.
- Episodic: for what we have done.
- Procedural: for the instruction of task memory.

Let's happy learning.

Khánh Phạm Đình, congratulations on completing Long-Term Agentic Memory With LangGraph! Khánh Phạm Đình, congratulations on completing Long-Term Agentic Memory With LangGraph!

[Turing Award Goes to A.I. Pioneers Andrew Barto and Richard Sutton]

1. What is new?
The Turing Award, often referred to as the "Nobel Prize of Computing," has been awarded to Andrew Barto and Richard Sutton, two pioneers in the field of artificial intelligence (A.I.). This recognition highlights their groundbreaking work in reinforcement learning, a subset of machine learning that involves training algorithms to make decisions based on trial and error. Their contributions have paved the way for significant advancements in A.I. research and applications.

2. What are the highlights?
Andrew Barto and Richard Sutton have been awarded the Turing Award for their work on reinforcement learning.
Their research has focused on developing algorithms that can learn from experience and adapt to new situations.
The award recognizes their contributions to the field of A.I., including the development of temporal difference learning and other reinforcement learning methods.
Barto and Sutton's work has had a significant impact on the development of autonomous systems, robotics, and other areas of A.I. research.
Their award-winning research has also led to the creation of new technologies and applications, including self-driving cars and personalized recommendation systems.

3. Why does it matter?
The awarding of the Turing Award to Andrew Barto and Richard Sutton matters because it highlights the importance of their work in advancing the field of A.I. Their research has the potential to transform numerous industries and aspects of our lives, from healthcare and finance to transportation and education. The recognition of their contributions also underscores the growing significance of A.I. in modern society, and the need for continued investment in research and development in this area. Furthermore, their work on reinforcement learning has the potential to lead to breakthroughs in areas such as decision-making, problem-solving, and human-computer interaction, which could have far-reaching implications for the future of technology and society.

Read more:
-->

Turing Award Goes to A.I. Pioneers Andrew Barto and Richard Sutton Andrew Barto and Richard Sutton developed reinforcement learning, a technique vital to chatbots like ChatGPT.

[AI Model Studying: ColPali - Efficient Document Retrieval with Vision Language Models]

1. What is New?
ColPali is an efficient Vision-Language Model (VLM) designed for document retrieval based solely on visual features. This model introduces a faster end-to-end pipeline that unifies textual embeddings and image-patch embeddings within the same feature space, ensuring alignment based on their factual meaning.

2. How Does It Work?
ColPali extends the capabilities of Paligemma-3B, a Vision-Language Model (VLM), leveraging its strong document understanding abilities. The core innovation lies in aligning the output embeddings of text and image tokens through multi-modal fine-tuning.

Encoding Text and Images & Defining Similarity
To quantify the similarity between a given query and a document image, we employ a Late Interaction (LI) mechanism. Given a query q and a document d:

The document image is divided into Nd patches and embedded using a VLM into a set of image-patch embedding vectors:
Ed = {Ed_1, Ed_2, ..., Ed_Nd}
The query is embedded into a set of Nq textual embedding vectors using an LLM:
Eq = {Eq_1, Eq_2, ..., Eq_Nq}
The Late Interaction (LI) function computes the similarity between the query and the document by identifying the maximum cosine similarity between the Nq textual embedding vectors and the Nd image-patch embedding vectors.
Contrastive Learning
Consider a batch of b relevant query-document pairs:
(q1, d1), (q2, d2), ..., (qb, db)

A contrastive loss is applied by selecting:

A positive pair (qi, di)
A hard negative pair (qi, dl), where dl is the document (from the set of b-1 documents excluding di) with the highest similarity to qi, as measured by the LI function.
The contrastive loss for each query qi is computed as follows:

Positive Score: s+ = LI(qi, di)
Negative Score: s- = LI(qi, dl)
Contrastive Loss:
L(qi) = - log(exp(s+) / (exp(s+) + exp(s-)))
Minimizing the contrastive loss is equivalent to maximizing the positive score while minimizing the negative score. This fine-tuning process forces the model to enhance the similarity between qi and its relevant document di while ensuring it can effectively discriminate against hard negatives that may otherwise appear highly similar.

3. What Are the Results?
ColPali significantly outperforms traditional text-based retrieval models. Benchmark evaluations show that it achieves an nDCG@5 score of 81.3, whereas traditional methods score between 65–75. This improvement of 6–16 points in retrieval accuracy underscores the model’s effectiveness in document indexing and retrieval tasks.

4. Why Does It Matter?
ColPali represents a significant leap forward in document retrieval, functioning more like a human reader rather than a simple text-based search tool. Unlike traditional methods that rely solely on textual content, ColPali comprehends:

Visual elements, such as images and their contents
Charts and diagrams, understanding the data they convey
Document layout, recognizing structural organization
Stylistic features, such as fonts and formatting
This holistic document understanding mimics human cognition, allowing ColPali to retrieve information with unprecedented accuracy and relevance.

📝 The paper: https://arxiv.org/abs/2407.01449
🗃️ The benchmark: https://huggingface.co/vidore
👀 The model: https://huggingface.co/vidore/colpali
💻 The benchmark code: https://github.com/illuin-tech/vidore-benchmark
💻 The training code: https://github.com/ManuelFay/colpali

[Build MLOps system on local with MLFlow library]

This tutorial teaches us critical knowledge in MLOps, specifically how to build and version a machine learning pipeline in MLFlow for regression tasks. The main content you will achieve includes:
1. Understand the core concept of MLFlow library.
2. Dive dive into analyzing an MLFlow recipe template for constructing MLFlow.
3. Learning how to investigate and trigger a Machine Learning pipeline.

- Link repo:
https://github.com/mlflow/recipes-examples

- Link diagram:
https://excalidraw.com/ =RTx1qDKIYkOPfN3SvhQZP,6xt5PMY-4-axP2-RD-a3Pg

- Video tutorial:
https://youtu.be/ddDQft7rirM

1. MLOps - Build machine learning pipeline by MLFlow recipe This tutorial teaches us critical knowledge in MLOps, specifically how to build and version a machine learning pipeline in MLFlow for regression tasks. The m...

[Deeply understanding How Transformer Works from Attention is all you need paper]

“Attention Is All You Need” is the most AI impactful paper in the 21st century, which first introduced the Transformer architecture. Let's learn how transformer-based large language models work in this DataQuest 4 event titled "Transformer Anatomy".

- Explore the architecture introduced in the “Attention Is All You Need” paper.
- Learn through intuitive explanations and code examples.

Watching and practicing hands-on lab in this free tutorial
https://youtu.be/XGU99IesqvE?si=npBitU6Jahkon00r

Link Colab notebook:
https://colab.research.google.com/drive/1_ceJB3D8rF1Z-naLqivVycYvYvo92Hx8?usp=sharing

DataQuest4_TransformerAnatomy.ipynb Colab notebook