Ngx?dZddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z m Z mZmZmZdefdZGdd eZGd d e ZdS) aaUtility for using SearxNG meta search API. SearxNG is a privacy-friendly free metasearch engine that aggregates results from `multiple search engines `_ and databases and supports the `OpenSearch `_ specification. More details on the installation instructions `here. <../../integrations/searx.html>`_ For the search API refer to https://docs.searxng.org/dev/search_api.html Quick Start ----------- In order to use this utility you need to provide the searx host. This can be done by passing the named parameter :attr:`searx_host ` or exporting the environment variable SEARX_HOST. Note: this is the only required parameter. Then create a searx search instance like this: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # when the host starts with `http` SSL is disabled and the connection # is assumed to be on a private network searx_host='http://self.hosted' search = SearxSearchWrapper(searx_host=searx_host) You can now use the ``search`` instance to query the searx API. Searching --------- Use the :meth:`run() ` and :meth:`results() ` methods to query the searx API. Other methods are available for convenience. :class:`SearxResults` is a convenience wrapper around the raw json result. Example usage of the ``run`` method to make a search: .. code-block:: python s.run(query="what is the best search engine?") Engine Parameters ----------------- You can pass any `accepted searx search API `_ parameters to the :py:class:`SearxSearchWrapper` instance. In the following example we are using the :attr:`engines ` and the ``language`` parameters: .. code-block:: python # assuming the searx host is set as above or exported as an env variable s = SearxSearchWrapper(engines=['google', 'bing'], language='es') Search Tips ----------- Searx offers a special `search syntax `_ that can also be used instead of passing engine parameters. For example the following query: .. code-block:: python s = SearxSearchWrapper("langchain library", engines=['github']) # can also be written as: s = SearxSearchWrapper("langchain library !github") # or even: s = SearxSearchWrapper("langchain library !gh") In some situations you might want to pass an extra string to the search query. For example when the `run()` method is called by an agent. The search suffix can also be used as a way to pass extra parameters to searx or the underlying search engines. .. code-block:: python # select the github engine and pass the search suffix s = SearchWrapper("langchain library", query_suffix="!gh") s = SearchWrapper("langchain library") # select github the conventional google search syntax s.run("large language models", query_suffix="site:github.com") *NOTE*: A search suffix can be defined on both the instance and the method level. The resulting query will be the concatenation of the two with the former taking precedence. See `SearxNG Configured Engines `_ and `SearxNG Search Syntax `_ for more details. Notes ----- This wrapper is based on the SearxNG fork https://github.com/searxng/searxng which is better maintained than the original Searx project and offers more features. Public searxNG instances often use a rate limiter for API usage, so you might want to use a self hosted instance and disable the rate limiter. If you are self-hosting an instance you can customize the rate limiter for your own network as described `here `_. For a list of public SearxNG instances see https://searx.space/ N)AnyDictListOptional)get_from_dict_or_env) BaseModel ConfigDictField PrivateAttrmodel_validatorreturnc dddS)Nenjson)languageformatrf/var/www/html/ai-engine/env/lib/python3.11/site-packages/langchain_community/utilities/searx_search.py_get_default_paramsrs / //rc|eZdZUdZdZeed<deffd ZdefdZe de fdZ e de fd Z xZ S) SearxResultsz,Dict like wrapper around search api results._datadatactj|}t|||_dS)zATake a raw result from Searx and make it into a dict like object.N)rloadssuper__init____dict__)selfr json_data __class__s rrzSearxResults.__init__s5Jt$$  ### rr c|jS)z$Text representation of searx result.)rr!s r__str__zSearxResults.__str__s zrc,|dS)zGSilence mypy for accessing this field. :meta private: resultsgetr%s rr(zSearxResults.resultss xx """rc,|dS)z#Helper accessor on the json result.answersr)r%s rr,zSearxResults.answerssxx """r)__name__ __module__ __qualname____doc__rstr__annotations__rr&propertyrr(r, __classcell__)r#s@rrrs66E3OOOS ####X#####X#####rrceZdZUdZeZeed<dZe ed<dZ e ed<e e Zeed<d Zeeed <gZeee ed <gZeee ed <dZee ed <dZeed<d Zeeed<edededefdZedZdedefdZ dedefdZ! d!de d eee d eee d ee dede f dZ" d"de d eee d ee dede f dZ# d!de ded eee d eee d ee dedeefdZ$ d"de ded eee d ee dedeef d Z%d S)#SearxSearchWrapperaWrapper for Searx API. To use you need to provide the searx host by passing the named parameter ``searx_host`` or exporting the environment variable ``SEARX_HOST``. In some situations you might want to disable SSL verification, for example if you are running searx locally. You can do this by passing the named parameter ``unsecure``. You can also pass the host url scheme as ``http`` to disable SSL. Example: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://localhost:8888") Example with SSL disabled: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # note the unsecure parameter is not needed if you pass the url scheme as # http searx = SearxSearchWrapper(searx_host="http://localhost:8888", unsecure=True) _resultr searx_hostFunsecure)default_factoryparamsNheadersengines categories query_suffix k aiosessionbefore)modevaluesr c|di}t}i|||d<|d}|rd||dd<|d}|rd||dd<t|dd}|dst d|d d |z}n|d rd |d <||d<|S)z?Validate that custom searx params are merged with default ones.r;r=,r>r8 SEARX_HOSThttpzRWarning: missing the url scheme on host ! assuming secure https:// zhttps://zhttp://Tr9)r*rjoinr startswithprint)clsrE user_paramsdefaultr=r>r8s rvalidate_paramsz"SearxSearchWrapper.validate_paramss,jj2.. %''5g55x**Y''  <*-((7*;*;F8 Y 'ZZ --  B-0XXj-A-AF8 \ *)&, MM $$V,, & 9+5999   $j0JJ  " "9 - - &!%F: )| rforbid)extractj|j|j||j }|jst d|jt|j}||_ |S)zActual request to searx API.r<r;verifySearx API returned an error: ) requestsr*r8r<r9ok ValueErrortextrr7)r!r; raw_resultress r_searx_api_queryz#SearxSearchWrapper._searx_api_querysh\ OL}$    } OC C CA E77 FFqueryrdc d|i}i|j||}|jr0t|jdkr|dxxd|jzz cc<t|tr&t|dkr|dxxd|zz cc<t|t r+t|dkrd||d<t|t r+t|dkrd||d<||}t|jdkr|jd} nLt|j dkr2dd|j d |j D} nd } | S) aoRun query through Searx API and parse results. You can pass any other params to the searx query API. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: str: The result of the query. Raises: ValueError: If an error occurred with the query. Example: This will make a query to the qwant engine: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://my.searx.host") searx.run("what is the weather in France ?", engine="qwant") # the same result can be achieved using the `!` syntax of searx # to select the engine using `query_suffix` searx.run("what is the weather in France ?", query_suffix="!qwant") qrrJrGr=r> c:g|]}|ddScontentrr).0rs r z*SearxSearchWrapper.run..d& U U U!y"!5!5 U U UrNNo good search result found) r;r?len isinstancer1listrKr^r,r(rA) r!rhr=r>r?rd_paramsr;r]torets rrunzSearxSearchWrapper.run%sP  6DK575f5   3T%6!7!7!!;!; 3KKK3!22 2KKK lC ( ( .S->->-B-B 3KKK3- -KKK gt $ $ 2W)9)9 # 1 1F9  j$ ' ' 8C OOa,?,?#&88J#7#7F< ##F++ s{  a  KNEE   ! !KK U Us{8TV8?T U U UVVEE1E rcKd|i}i|j||}|jr0t|jdkr|dxxd|jzz cc<t|tr&t|dkr|dxxd|zz cc<t|t r+t|dkrd||d<||d{V}t|jdkr|jd}nLt|j dkr2dd|j d|j D}nd }|S) z Asynchronously version of `run`.rjrrJrGr=Nrkc:g|]}|ddSrmr)ros rrrz+SearxSearchWrapper.arun..rsrrt) r;r?rurvr1rwrKrgr,r(rA) r!rhr=r?rdrxr;r]rys rarunzSearxSearchWrapper.arunjsx  6DK575f5   3T%6!7!7!!;!; 3KKK3!22 2KKK lC ( ( .S->->-B-B 3KKK3- -KKK gt $ $ 2W)9)9 # 1 1F9 **622222222 s{  a  KNEE   ! !KK U Us{8TV8?T U U UVVEE1E r num_resultsc d|i}i|j||}|jr0t|jdkr|dxxd|jzz cc<t|tr&t|dkr|dxxd|zz cc<t|t r+t|dkrd||d<t|t r+t|dkrd||d<||jd|} t| dkrdd igSd | DS) a$Run query through Searx API and returns the results with metadata. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. num_results: Limit the number of results to return. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: Dict with the following keys: { snippet: The description of the result. title: The title of the result. link: The link to the result. engines: The engines used for the result. category: Searx category of the result. } rjrrJrGr=r>NResultNo good Search Result was foundcvg|]6}|dd|d|d|d|dd7Srnrtitleurlr=category)snippetrlinkr=rr)rprfs rrrz.SearxSearchWrapper.results..`     "::i44u !),":.       r) r;r?rurvr1rwrKr^r() r!rhr~r=r>r?rdrxr;r(s rr(zSearxSearchWrapper.resultssw>  6DK575f5   3T%6!7!7!!;!; 3KKK3!22 2KKK lC ( ( .S->->-B-B 3KKK3- -KKK gt $ $ 2W)9)9 # 1 1F9  j$ ' ' 8C OOa,?,?#&88J#7#7F< ''//7  E w<<1  @AB B   "    rc$Kd|i}i|j||}|jr0t|jdkr|dxxd|jzz cc<t|tr&t|dkr|dxxd|zz cc<t|t r+t|dkrd||d<||d{Vjd|}t|dkrddigSd |DS) zdAsynchronously query with json results. Uses aiohttp. See `results` for more info. rjrrJrGr=Nrrcvg|]6}|dd|d|d|d|dd7Srr)rs rrrz/SearxSearchWrapper.aresults..rr) r;r?rurvr1rwrKrgr() r!rhr~r=r?rdrxr;r(s raresultszSearxSearchWrapper.aresultssU  6DK575f5   3T%6!7!7!!;!; 3KKK3!22 2KKK lC ( ( .S->->-B-B 3KKK3- -KKK gt $ $ 2W)9)9 # 1 1F9 //77777777@+N w<<1  @AB B   "    r)NNr)Nr)&r-r.r/r0r r7rr2r8r1r9boolr rr;dictr<rr=rr>r?rAintrBrr classmethodrrQr model_configr^rgrzr}r(rrrrr6r6sb6(KMMG\)))JHd5)<===FD==="GXd^"""#%GXd3i %%%&(Jc#((("$L(3-$$$ AsKKK $J $$$_(###Tc[$#6:L t     d|>(,*.&( CCC$s)$CT#Y' C sm C  C CCCCP(,&( !!!$s)$!sm !  ! !!!!N(,*.&( 7 7 7 7 $s)$ 7 T#Y' 7 sm 7 7  d7 7 7 7 z(,&( $ $ $ $ $s)$ $ sm $  $  d$ $ $ $ $ $ rr6)r0rtypingrrrrrarXlangchain_core.utilsrpydanticrr r r r rrrr6rrrrs.B ,,,,,,,,,,,,5555550T0000#####4###:x x x x x x x x x x r