o
}h @ s d dl Z d dlZd dlZd dlZd dlmZmZmZ d dlm Z d dl
m Z d dl m
Z
d dlmZ ejejd eeZe G dd d e
ZG d
d d ZdS ) N)DictAnyOptional) load_dotenv)
ChatOllama)BaseCallbackHandler)retrieve_semantic_docs)levelc @ s" e Zd Zdd ZdefddZdS )StreamingToSocketHandlerc C s
|| _ d S N
send_token_fn)selfr
r &/var/www/observatorio/agentic_rag_4.py__init__ s
z!StreamingToSocketHandler.__init__tokenc K s | | d S r r )r r kwargsr r r on_llm_new_token s z)StreamingToSocketHandler.on_llm_new_tokenN)__name__
__module__ __qualname__r strr r r r r r
s r
c @ sj e Zd ZdZddefddZdedeeef fdd Zd
edefd d Z d
ede
eeef fddZ dS )SimpleDeepSeekRAGzFAgentic RAG system for DeepSeek-R1 with fixed search + answer behaviordeepseek-r1:8b
model_namec C sF t dd | _t|| jddddt|gd| _td| d S ) N OLLAMA_HOSTzhttp://localhost:11434g ?x i T)modelbase_url temperaturetimeout num_predict streaming callbacksz Initialized DeepSeek RAG with ) osgetenvstrip ollama_hostr r
llmloggerinfo)r stream_funcr r r r r ! s zSimpleDeepSeekRAG.__init__questionreturnc
C s zd}d| d}| j |j }| |}td| zt|}tdt|t r0t
|nd d W n t yS } zt d| g }W Y d }~nd }~ww d }g } d
}
|rt|t sd|g}d }t
|dD ]D\} } t| tr| d }
| d
}| dpd}t
||kr|d | d }|
|d 7 }
|d|
d| d| d7 }| |
|d qkd| d| d}| j |j }| |}d|r|n|||t
|| |
dW S t y } zd t| }t | d |d d W Y d }~S d }~ww ) NiX z
Based on this question, and considering that this is a chat strictly about Brazilian legal decisions of 2024 on elections.
What should I search for in a document database using the KNN search?
Question: z
Provide only the main search terms in Portuguese, separated by spaces.
Write the query inside tags.
zGenerated search query: zSearch returned: z resultszSearch failed: zNo documents found. zFOUND DOCUMENTS:
numero_unicodata_publicacao
conteudo_htmlz
No contentz...z
z
Document ID: z
Date: z
Content:
)iddateu
Based on the search results below, answer this question **in Brazilian Portuguese (português do Brasil)**.
Question: z*
Search Results:
a
IMPORTANT RULES:
- Only use information from the search results above
- Say everything you found in the document, but priorize answering the question.
- Be specific about what the documents say about the topic
- You must mention the document IDs in your answer
- When reasoning about the documents, make sure to open and close the tags
- Explain clearly and directly, without additional labels or markings
- ONLY send the final answer inside tags in this exact format:
your detailed answer based on search results, written in Brazilian Portuguese
T)successresponse
raw_output search_querysearch_results_countsources
total_contentzQuery failed: F)r7 errorr8 )r) invokecontentr' _extract_search_termsr* r+ r
isinstancelistlen Exceptionr> enumeratedictgetappend
_extract_jsonr )r r- CHAR_CAP
search_promptraw_search_responser: search_resultseformatted_results source_listr= idocdoc_iddoc_dater@
answer_promptr8
json_response error_msgr r r query/ sx
(
zSimpleDeepSeekRAG.query raw_responsec
C s z_t jdd|t jd }|rt|dk r?t d|t j}|r?|d}|d}dd |D }|r.*?r0 flags z(.*?)r/ r4 c S s4 g | ]}| d r|dd dd qS )z- N(r )r'
startswithsplit.0liner r r
s z;SimpleDeepSeekRAG._extract_search_terms.. documentoc S s g | ]
}| r| qS r )r' rc r r r rf s z\s+r d z Search term extraction failed: ) resubDOTALLr' rD searchgrouprb joinrE r* r> ) r rZ
clean_text think_matchthinking_contentlinesterms
first_linerO r r r rA s.
z'SimpleDeepSeekRAG._extract_search_termstextc
C s zMt jdd|t jd }ddg}|D ]7}t ||t j}|D ]*}z t| }t|tr>d|v s6d|v r>|W W S W q tj yI Y q w qW d S t
yh } zt d| W Y d }~d S d }~ww ) Nr[ r0 r\ z \{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}z\{\s*"answer"[^}]+\}answerr8 zJSON extraction failed: )
rk rl rm r' findalljsonloadsrB rG JSONDecodeErrorrE r* r> ) r rw rq patternspatternmatchesmatchparsedrO r r r rJ s. z SimpleDeepSeekRAG._extract_jsonN)r ) r r r __doc__r r r r rY rA r rJ r r r r r s b" r )rz r% loggingrk typingr r r dotenvr langchain_community.chat_modelsr langchain_core.callbacks.baser data_searchr basicConfigINFO getLoggerr r* r
r r r r r s