Ng1ddlZddlZddlZddlmZmZddlZddlmZddl m Z ej dZ ej dZ Gdde ZdS) N)ListTuple)Document) BaseLoaderzBV\w+zav[0-9]+c veZdZdZ d deedededefdZdeefd Zd ede ee ffd Z d S)BiliBiliLoaderz9 Load fetching transcripts from BiliBili videos. video_urlssessdatabili_jctbuvid3c||_d|_ ddlm}n#t$rt dwxYw|r#|r#|r#|||||_dSdSdSdS)a Initialize the loader with BiliBili video URLs and authentication cookies. if no authentication cookies are provided, the loader can't get transcripts and will only fetch videos info. Args: video_urls (List[str]): List of BiliBili video URLs. sessdata (str): SESSDATA cookie value for authentication. bili_jct (str): BILI_JCT cookie value for authentication. buvid3 (str): BUVI3 cookie value for authentication. Nr)videoTrequests package not found, please install it with `pip install bilibili-api-python`)r r r )r credential bilibili_apir ImportError Credential)selfr r r r rs i/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/document_loaders/bilibili.py__init__zBiliBiliLoader.__init__s$%  * * * * * * *   4     V #..!HV/DOOO      s1returncg}|jD]@}||\}}t||}||A|S)z Load and return a list of documents containing video transcripts. Returns: List[Document]: List of Document objects transcripts and metadata. ) page_contentmetadata)r _get_bilibili_subs_and_inforappend)rresultsurl transcript video_infodocs rloadzBiliBiliLoader.load5s^?  C%)%E%Ec%J%J "J  ZHHHC NN3    rct|} ddlm}m}n#t $rt dwxYw|r/|||j}nrt|}|rD|t|dd|j}ntd||| }| d |i|jsd |fS|||d }|d g} | r| dd d } | dsd| z} t#j| } | jdkrgt'j| jdg} dd| D} d|dd|dd| }||fSt/jd|d| jnt/jd|dd |fS)zU Retrieve video information and transcript for a given BiliBili URL. r)syncrr)bvidrN)aidrz(Unable to find a valid video ID in URL: rr cid subtitles subtitle_urlhttpzhttps:body cg|] }|d S)content).0cs r z>BiliBiliLoader._get_bilibili_subs_and_info..ks*P*P*PA1Y<*P*P*Pr$z Video Title: titlez, description: descz Transcript: zFailed to fetch subtitles for z. HTTP Status Code: zNo subtitles found for video: z. Returning empty transcript.) BV_PATTERNsearchrr&rrVideogroupr AV_PATTERNint ValueErrorget_infoupdate get_subtitleget startswithrequests status_codejsonloadsr2joinwarningswarn)rrr'r&rvr)r!subsub_listsub_urlresponseraw_sub_titlesraw_transcriptraw_transcript_with_meta_infos rrz*BiliBiliLoader._get_bilibili_subs_and_infoDs  %%  0 0 0 0 0 0 0 0 0   4    S $/ JJAA##C((C SKKC ABB$8$8T_KUU !QC!Q!QRRRT!**,,'' 5#,''' "z> !d1>>*U"3445577;++  qkoonb99G%%f-- -"W,|G,,H#s**!%H,r_s ------@@@@@@RZ ! ! RZ $ $ nnnnnZnnnnnr$