NgR R d dl Z d dlZd dlmZ d dlmZ d dlmZmZm Z m
Z
m Z m Z d dl
mZ d dlmZmZ d dlmZ d dlmZ d d lmZ d d
lmZmZmZm Z m Z m Z d efd Z d
e e ef d e!fdZ" dde de#de#dede
e e ef d e!f dZ$de de d efdZ% G d dee Z&dS ) N)deepcopy)Path)AnyDictListOptionalSequenceUnion)warn_deprecated) AgentAction AgentFinish)BaseCallbackHandler) LLMResult guard_import)BaseMetadataCallbackHandler flatten_dict hash_string
import_pandas import_spacyimport_textstatreturnc t d S )zJImport the wandb python package and raise an error if it is not installed.wandbr h/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/callbacks/wandb_callback.py import_wandbr s r json_pathc ~ t | d 5 }t j | }ddd n # 1 swxY w Y |S )zLoad json file to a dictionary.
Parameters:
json_path (str): The path to the json file.
Returns:
(dict): The dictionary representation of the json file.
rN)openjsonload)r fdatas r load_json_to_dictr' sx
i y|| Ks 266Ttextcomplexity_metrics visualizenlp
output_dirc i }t }t }t }|rwi d| | d| | d| | d| | d| | d| | d| | d|
| d | | d
| | d |
| d | | d
| | d| | d| | d| | } | | |rA|r>|; || }
|j |
ddd } t- |t/ d| dz } | dd | |j |
ddd }
t- |t/ d | dz }| dd |
| t7 | | t7 | d }| | |S ) a Analyze text using textstat and spacy.
Parameters:
text (str): The text to analyze.
complexity_metrics (bool): Whether to compute complexity metrics.
visualize (bool): Whether to visualize the text.
nlp (spacy.lang): The spacy language model to use for visualization.
output_dir (str): The directory to save the visualization files to.
Returns:
(dict): A dictionary containing the complexity metrics and visualization
files serialized in a wandb.Html element.
flesch_reading_easeflesch_kincaid_grade
smog_indexcoleman_liau_indexautomated_readability_index dale_chall_readability_scoredifficult_wordslinsear_write_formula gunning_fog
text_standardfernandez_huertaszigriszt_pazosgutierrez_polinicrawfordgulpease_indexosmanNdepFT)stylejupyterpagezdep-z.htmlwzutf-8)encodingentzent-)dependency_treeentities) r r r r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r; r< r= updatedisplacyrenderr r r" writeHtmlstr)r( r) r* r+ r, resptextstatr spacytext_complexity_metricsdocdep_outdep_output_pathent_outent_output_pathtext_visualizationss r analyze_textrW * s- ( D H NNE NNE -#
!8#?#?#E#E#
"H$A$A$$G$G#
(--d33#
!("="=d"C"C #
*8+O+OPT+U+U #
+H,Q,QRV,W,W
#
x 7 7 = =#
$X%C%CD%I%I#
8//55#
X 3 3D 9 9#
9 9$ ? ?#
x 7 7 = =#
9 9$ ? ?#
))$// #
h 5 5d ; ; #
X^^D))!#
$
+,,, )S )Z 3c$ii.''5%d'SS z;}d}}+E+E+OPP S7 3399'BBB.''5%d'SS z;}d}}+E+E+OPP S7 3399'BBB %zz#o*>*> ? ?
3#7#788
'((( Kr prompt
generationc t }| dd }| dd }| d| d| dd S )zConstruct an html element from a prompt and a generation.
Parameters:
prompt (str): The prompt.
generation (str): The generation.
Returns:
(wandb.Html): The html element.
z
z
z>:
z"
F)inject)r replacerK )rX rY r formatted_promptformatted_generations r )construct_html_from_prompt_and_generationr` k s}
NNE ~~dF33%--dF;; :: -
r c e Zd ZdZ d1dee dee dee dee d ee d
ee d ee d ed
ededdf fd
Zde fdZ
de ee f de e de ddfdZ
dede ddfdZdede ddfdZdede ddfdZde ee f d e ee f de ddfd Zd e ee f de ddfd Zdede ddfd Zde ee f d!ede ddfd"Zd#e de ddfd$Zdede ddfd%Zd&ede ddfd'Zd(ede ddfd)Zd*e de de fd+Z de fd,Z d2d.e d/ed(edee dee dee dee d ee d
ee d ee d ee d
ee ddfd0Z xZ S )3WandbCallbackHandlera Callback Handler that logs to Weights and Biases.
Parameters:
job_type (str): The type of job.
project (str): The project to log to.
entity (str): The entity to log to.
tags (list): The tags to log.
group (str): The group to log to.
name (str): The name of the run.
notes (str): The notes to log.
visualize (bool): Whether to visualize the run.
complexity_metrics (bool): Whether to log complexity metrics.
stream_logs (bool): Whether to stream callback actions to W&B
This handler will utilize the associated callback method called and formats
the input of each callback function with metadata regarding the state of LLM run,
and adds the response to the list of records for both the {method}_records and
action. It then logs the response using the run.log() method to Weights and Biases.
Nlangchain_callback_demoFjob_typeprojectentitytagsgroupnamenotesr* r) stream_logsr c t } t t t } t || _ || _ || _ || _ || _
|| _ || _ || _
| | _ |
| _ t! j | _ | | j | j | j | j | j
| j | j | _ d}
| |
d g | _ g | _ | | _ || _
| d | _ t5 dddd d
S ) z Initialize callback handler.)rd re rf rg rh ri rj zDEPRECATION: The `WandbCallbackHandler` will soon be deprecated in favor of the `WandbTracer`. Please update your code to use the `WandbTracer` instead.F)repeaten_core_web_smz0.3.8zPlease use the WeaveTracer instead of the WandbCallbackHandler. The WeaveTracer is a more flexible and powerful tool for logging and tracing your LangChain callables.Find more information at https://weave-docs.wandb.ai/guides/integrations/langchainzPlease instantiate the WeaveTracer from weave.integrations.langchain import WeaveTracer .For autologging simply use weave.init() and log all traces from your LangChain callables.)pendingmessage alternativeN)r r r r super__init__rd re rf rg rh ri rj r* r) rk tempfileTemporaryDirectorytemp_dirinitruntermwarncallback_columnsaction_recordsr$ r+ r )selfrd re rf rg rh ri rj r* r) rk r rO warning __class__s r rs z WandbCallbackHandler.__init__ sk
""4 & 355
::]L;**
') $&"4 ":: .// e 1
r c $ d | j D S )Nc i | ]}|d S )Nr ).0ks r
z3WandbCallbackHandler._init_resp.. s 777A4777r )rz )r| s r
_init_respz WandbCallbackHandler._init_resp s 77!67777r
serializedpromptskwargsc * | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| ddi | t | | | |D ]k}t | }||d< | j | | j
| | j r| j
| ldS )zRun when LLM starts. action on_llm_startr N)step
llm_startsstartsr rG r get_custom_callback_metar on_llm_start_recordsappendr{ rk rx log)r| r r r rM rX prompt_resps r r z!WandbCallbackHandler.on_llm_start s
Q 1 q X~./// L,,--- D1133444 * *F"4..K%+K " % , ,[ 9 9 9 & &{ 3 3 3
* [)))
* *r tokenc | xj dz
c_ | xj dz
c_ | }| d|d | | | j | | j | | j r | j
| dS dS )z#Run when LLM generates a new token.r on_llm_new_token)r r N) r llm_streamsr rG r on_llm_token_recordsr r{ rk rx r )r| r r rM s r r z%WandbCallbackHandler.on_llm_new_token s Q A 1EBBCCC D1133444 !((... ""4((( HLL r responsec 4 | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| ddi | t |j pi | | |j D ]}|D ]}t | }| t |
| t |j | j
| j | j | j j | j | | j | | j r| j | dS )zRun when LLM ends running.r r
on_llm_end)r) r* r+ r, N)r llm_endsendsr rG r
llm_outputr generationsr dictrW r( r) r* r+ rv ri on_llm_end_recordsr r{ rk rx r )r| r r rM r rY generation_resps r r z WandbCallbackHandler.on_llm_end s Q
Q X|,--- L!4!:;;<<< D1133444#/ 2 2K )
2
2
"*4.. &&|JOO4E4E'F'FGGG && "+/+B"&. H#'=#5 '..???#**?;;;# 2HLL 111
2 2 2r errorc F | xj dz
c_ | xj dz
c_ dS )zRun when LLM errors.r Nr errorsr| r r s r on_llm_errorz!WandbCallbackHandler.on_llm_error& % Q q r inputsc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| ddi | t | | | |d }t |t rmt | }||d< | j
| | j | | j
r | j | dS dS t |t r|D ]{}t | }| | | j
| | j | | j
r| j | |dS t# d )z Run when chain starts running.r r on_chain_startinputz Unexpected data format provided!N)r chain_startsr r rG r r
isinstancerL r on_chain_start_recordsr r{ rk rx r list
ValueError)r| r r r rM chain_input
input_respinps r r z#WandbCallbackHandler.on_chain_start+ s
Q Q q X /0111 L,,--- D1133444 Wo k3 ' ' A!$ J"-Jw ' . .z : : : & &z 2 2 2
) Z(((((
)
)
T
*
* A"
-
- %d^^
!!#&&&+22:>>>#**:666# -HLL,,,
-
- ?@@ @r outputsc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|d d | | | j | | j | | j r | j
| dS dS )z Run when chain ends running.r on_chain_endoutput)r r N) r
chain_endsr r rG r on_chain_end_recordsr r{ rk rx r )r| r r rM s r r z!WandbCallbackHandler.on_chain_endL s Q 1 Q ~'(:KLLMMM D1133444 !((... ""4((( HLL r c F | xj dz
c_ | xj dz
c_ dS )zRun when chain errors.r Nr r s r on_chain_errorz#WandbCallbackHandler.on_chain_error[ r r input_strc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|d | t | | | | j | | j | | j
r | j | dS dS )z Run when tool starts running.r
on_tool_start)r r N)
r tool_startsr r rG r r on_tool_start_recordsr r{ rk rx r )r| r r r rM s r r z"WandbCallbackHandler.on_tool_start` s
Q A q YGGHHH L,,--- D1133444 "))$/// ""4((( HLL r r c t | }| xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|d | | | j | | j | | j
r | j | dS dS )zRun when tool ends running.r on_tool_end)r r N)
rL r tool_endsr r rG r on_tool_end_recordsr r{ rk rx r )r| r r rM s r r z WandbCallbackHandler.on_tool_endr s V Q ! Q }??@@@ D1133444 ''--- ""4((( HLL r c F | xj dz
c_ | xj dz
c_ dS )zRun when tool errors.r Nr r s r
on_tool_errorz"WandbCallbackHandler.on_tool_error r r r( c | xj dz
c_ | xj dz
c_ | }| d|d | | | j | | j | | j r | j
| dS dS )z+
Run when agent is ending.
r on_text)r r( N) r text_ctrr rG r on_text_recordsr r{ rk rx r )r| r( r rM s r r z WandbCallbackHandler.on_text s
Q
y$77888 D1133444 ##D))) ""4((( HLL r finishc | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|j d |j d | | | j | | j
| | j r | j | dS dS )z Run when agent ends running.r on_agent_finishr )r r r N)
r
agent_endsr r rG
return_valuesr r on_agent_finish_recordsr r{ rk rx )r| r r rM s r r z$WandbCallbackHandler.on_agent_finish s Q 1 Q + .x8 z
D1133444 $++D111 ""4((( HLL r r c | xj dz
c_ | xj dz
c_ | xj dz
c_ | }| d|j |j |j d | | | j
| | j
| | j r | j
| dS dS )zRun on agent action.r on_agent_action)r tool
tool_inputr N)r r r r rG r r r r on_agent_action_recordsr r{ rk rx )r| r r rM s r r z$WandbCallbackHandler.on_agent_action s Q A q + $ / z
D1133444 $++D111 ""4((( HLL r c t }| | j }| | j }|g d d ddid }g }g }| j rg d}| j rddg}|g d |z |z d d
d d d }| ||gd }|d
d g d d |d< |S )z=Create a dataframe with all the information from the session.)r r ri r )axisr prompt_step)r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r; r< r= rE rF )r r( token_usage_total_tokenstoken_usage_prompt_tokens token_usage_completion_tokens output_stepr )r r( r c : t | d | d S )Nr r )r` )rows r zBWandbCallbackHandler._create_session_analysis_df.. s AIH
r chat_html)
r DataFramer r dropnarenamer) r* concatapply) r| pdon_llm_start_records_dfon_llm_end_records_dfllm_input_prompts_dfcomplexity_metrics_columnsvisualizations_columnsllm_outputs_dfsession_analysis_dfs r _create_session_analysis_dfz0WandbCallbackHandler._create_session_analysis_df so
__"$,,t/H"I"I " T-D E E
$$?$?$? @
VV^^
VV]+!V
4
4
&("!# " * * * && > E&7%D "
" -- ))
VV^^
V]H==AV
F
F !ii)=~(NUVi W W+>
!,
%
K( #"r Tlangchain_assetresetc
` t }
t }| |
| j }| | }| j ||d |r^t | j j
d }| dd }| |d | |d |
| | t | t! | |_ n# t$ $ rN | | | t | t! | |_ Y nAt( $ r5}t+ d t+ t- | Y d
}~nd
}~ww xY w| j | |s|rF| j | j | |rq| |r|n| j |r|n| j |r|n| j |r|n| j |r|n| j | r| n| j
|
r|
n| j! | r| n| j" | r| n| j# d
S d
S ) aS Flush the tracker and reset the session.
Args:
langchain_asset: The langchain asset to save.
reset: Whether to reset the session.
finish: Whether to finish the run.
job_type: The job type.
project: The project.
entity: The entity.
tags: The tags.
group: The group.
name: The name.
notes: The notes.
visualize: Whether to visualize.
complexity_metrics: Whether to compute complexity metrics.
Returns:
None
) dataframe)r{ session_analysisz
model.jsonmodel)ri typer{ )ri r zCould not save model.N) rd re rf rg rh ri rj r* r) )$r r Tabler r{ r rx r r rv ri Artifactaddsaveadd_filerL r' metadatar
save_agentNotImplementedErrorprintrepr log_artifactr cleanupreset_callback_metars rd re rf rg rh rj r* r) )r| r r r rd re rf rg rh ri rj r* r) r r action_records_tablesession_analysis_tablelangchain_asset_pathmodel_artifactes r
flush_trackerz"WandbCallbackHandler.flush_tracker s D __ ${{R\\$BU5V5V{ W W !&6688 "- "
"
"6$:
2#'
(:L#I#I "^^w^ G GN 3:J K K K 5U %TNU
U U U U U U U Ur rb )TTNN)'r# rt copyr pathlibr typingr r r r r r
langchain_core._apir langchain_core.agentsr r
langchain_core.callbacksr langchain_core.outputsr langchain_core.utilsr #langchain_community.callbacks.utilsr r r r r r r rL r r' r rW r` rb r r r r s: = = = = = = = = = = = = = = = = / / / / / / : : : : : : : : 8 8 8 8 8 8 , , , , , , - - - - - - !c ! ! ! !
sDy!1 d $-1 > >
> > >
>
sDy)* >
> > > >Bc s s 4P P P P P68K P P P P Pr