�ɲɾ�����ӯ�����һ��ˣ��������С���˴��ͣ�������P���ҹ��ñ˽��ά�Բ��������˸߸ԣ�������ơ��ҹ��ñ�����ά�Բ���ˡ���˳^�ӣ������ӡ� ���ͯj�ӣ��ƺ���ӣ� ? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!PK{0] [/DD testing.pyonu[ \[c@sdZdZddlZddlZddlZddlZddlmZddlmZddl m Z m Z m Z m Z mZddlmZeZdejfd YZd efd YZd efd YZdefdYZdZdS(sHelper classes for tests.tMITiN(tTestCase(t BeautifulSoup(tCharsetMetaAttributeValuetCommenttContentMetaAttributeValuetDoctypet SoupStrainer(tHTMLParserTreeBuildertSoupTestcBs>eZedZdZdZddZdZRS(cCstS(N(tdefault_builder(tself((s//usr/lib/python2.7/site-packages/bs4/testing.pyR scKs(|jd|j}t|d||S(s*Build a Beautiful Soup object from markup.tbuilder(tpopR R(R tmarkuptkwargsR ((s//usr/lib/python2.7/site-packages/bs4/testing.pytsoup scCs|jj|S(s[Turn an HTML fragment into a document. The details depend on the builder. (R ttest_fragment_to_document(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyt document_for%scCsS|j}t|d|}|dkr0|}n|j|j|j|dS(NR (R RtNonet assertEqualtdecodeR(R tto_parsetcompare_parsed_toR tobj((s//usr/lib/python2.7/site-packages/bs4/testing.pytassertSoupEquals,s    cCsSd}xF|jD];}|rE|j||j|j||jn|}qWdS(syEnsure that next_element and previous_element are properly set for all descendants of the given element. N(Rt descendantsRt next_elementtprevious_element(R telementtearlierte((s//usr/lib/python2.7/site-packages/bs4/testing.pytassertConnectedness4s N( t__name__t __module__tpropertyR RRRRR (((s//usr/lib/python2.7/site-packages/bs4/testing.pyR s    tHTMLTreeBuilderSmokeTestcBseZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZ dZ!d Z"d!Z#d"Z$d#Zd$Z%d%Z&d&Z'd'Z(d(Z)d)Z*d*Z+d+Z,d,Z-d-Z.d.Z/d/Z0d0Z1d1Z2d2Z3d3Z4d4Z5d5Z6d6Z7d7Z8d8Z9RS(9sCA basic test of a treebuilder's competence. Any HTML treebuilder, present or future, should be able to pass these tests. With invalid markup, there's room for interpretation, and different parsers can handle it differently. But with the markup in these tests, there's not much room for interpretation. cCs|xuddddddddd d d d d dddddgD]7}|jd}|j|}|jt|jq=WdS(smVerify that all HTML4 and HTML5 empty element (aka void element) tags are handled correctly. tareatbasetbrtcoltembedthrtimgtinputtkeygentlinktmenuitemtmetatparamtsourcettracktwbrtspacertframetN(Rtnew_tagRtTruetis_empty_element(R tnameRR8((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_empty_element_tagsIs 0cCsc|jd}tj|d}tj|}|j|jt|j|j|jdS(Ns fooi(RtpickletdumpstloadsRt __class__RR(R ttreetdumpedtloaded((s//usr/lib/python2.7/site-packages/bs4/testing.pyt!test_pickle_and_unpickle_identityUs cCs|j|\}}|jd}|j|jt|j|||jt|t| ||j|jjdddS(s8Assert that a given doctype string is handled correctly.itfooN(t_document_with_doctypetcontentsRR@Rtstrtlentp(R tdoctype_fragmentt doctype_strRtdoctype((s//usr/lib/python2.7/site-packages/bs4/testing.pytassertDoctypeHandled^s   cCs-d|}|d}|j|}||fS(s5Generate and parse a document with the given doctype.s s

foo

(R(R RKRMRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyRFls  cCs|jd|jddS(s?Make sure normal, everyday HTML doctypes are handled correctly.thtmls4html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"N(RN(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_normal_doctypesss cCs6|jd}|jd}|jd|jdS(Ns iR7(RRGRtstrip(R RRM((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_empty_doctypeys cCsd}|j|dS(Nsnhtml PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"(RN(R RM((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_public_doctype_with_url~scCs|jddS(Ns$foo SYSTEM "http://www.example.com/"(RN(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_system_doctypescCs|jddS(Ns#xsl:stylesheet SYSTEM "htmlent.dtd"(RN(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_namespaced_system_doctypescCs|jddS(Ns#xsl:stylesheet PUBLIC "htmlent.dtd"(RN(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_namespaced_public_doctypescCsJd}|j|}|j|jdjdd|jdddS(sJA real XHTML document should come out more or less the same as it went in.s Hello. Goodbye. sutf-8s R7N(RRtencodetreplace(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_real_xhtml_documents cCs8d}|j|}|jdt|jddS(stWhen a namespaced XML document is parsed as HTML it should be treated as HTML with weird tag names. s.contentisns1:fooN(RRRItfind_all(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_namespaced_htmlscCs]d}|j|}|j||jd}|j|}|j||jddS(Nustutf8(RRRRW(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_processing_instructions cCstj|jdS(sMake sure you can copy the tree builder. This is important because the builder is part of a BeautifulSoup object, and we want to be able to copy that. N(tcopytdeepcopyR (R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt test_deepcopyscCs?|jd}|j|jj|jt|jddS(sA

tag is never designated as an empty-element tag. Even if the markup shows it as an empty-element tag, it shouldn't be presented that way. s

s

N(Rt assertFalseRJR:RRH(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyt!test_p_tag_is_never_empty_elementscCs4|jdd|jdd|jdddS(sA tag that's not closed by the end of the document should be closed. This applies to all tags except empty-element tags. s

s

sss
s
N(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_unclosed_tags_get_closedscCs?|jd}|j|jj|jt|jddS(sA
tag is designated as an empty-element tag. Some parsers treat

as one
tag, some parsers as two tags, but it should always be an empty-element tag. s

s
N(Rt assertTrueR'R:RRH(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyt#test_br_is_always_empty_element_tagscCs|jddS(Ns(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_nested_formatting_elementsscCs6d}|j|}|jd|jdddS(Ns Ordinary HEAD element test Hello, world! stext/javascripttscriptttype(RRtfind(R ROR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_double_heads cCsd}|j||j|}|jdd}|j|jt|jdd}|j||j|jdd}|j||jdS(Ns

foobaz

ttexttfoobarREtbaz(RRRiRR@RRR(R RRtcommentRERm((s//usr/lib/python2.7/site-packages/bs4/testing.pyt test_comments cCsd}d}|j||j||j|}|j|jj||j|}|j|jj||jd}|j|jjddS(sWhitespace must be preserved in
 and sN(RRRtpretprettifyttextarea(Rt
pre_markupttextarea_markupR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt-test_preserved_whitespace_in_pre_and_textareas

cCs=d}|j|d}|j|d}|j|dS(s+Inline elements can be nested indefinitely.sInside a B tags!

A nested tag

s/

A doubly nested tag

N(R(R tb_tagt nested_b_tagtdouble_nested_b_tag((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_nested_inline_elements s   cCsK|jd}|j}|j|jjjd|j|jjddS(sBlock elements can be nested.s*

Foo

tFooN(Rt blockquoteRRJtbtstring(R RR{((s//usr/lib/python2.7/site-packages/bs4/testing.pyt test_nested_block_level_elementss cCs'd}|j|d|jddS(s$One table can go inside another one.s[sh
Here's another table:
foo
Here's another table:
foo
s{
Foo
Bar
Baz
N(R(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_correctly_nested_tabless cCs6d}|j|}|jdg|jjddS(Ns1
tcsstclass(RRtdiv(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt(test_deeply_nested_multivalued_attribute0scCs6d}|j|}|jddg|jddS(NstaR|R(RRRO(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt"test_multivalued_attribute_on_html8scCs|jdddS(Nss(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt3test_angle_brackets_in_attribute_values_are_escaped@scCs|jdddS(Nu$

• AT&T is in the s&p 500

u)

• AT&T is in the s&p 500

(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt3test_strings_resembling_character_entity_referencesCscCs/d}|j|}|jd|jjdS(Ns%

“Hello” -☃

u“Hello” -☃(Rt assertEqualsRJR}(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt*test_entities_in_foreign_document_encodingKscCsJd}|jd||jd||jd||jd|dS(Nu

s

s

s

s

(R(R texpect((s//usr/lib/python2.7/site-packages/bs4/testing.pyt0test_entities_in_attributes_converted_to_unicodeWs cCsJd}|jd||jd||jd||jd|dS(Nu

piñata

s

piñata

s

piñata

s

piñata

s

piñata

(R(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyt*test_entities_in_text_converted_to_unicode^s cCs|jdddS(Ns#

I said "good day!"

s

I said "good day!"

(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt,test_quot_entity_converted_to_quotation_markes cCs:d}|jd||jd||jd|dS(Nu�s�s�s �(R(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_out_of_range_entityiscCsR|jd}|jd|jjjj|jd|jj|j|dS(sDMostly to prevent a recurrence of a bug in the html5lib treebuilder.s!

foo

RJN(RRth2R}RR;RJR (R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_multipart_stringsoscCs$|jdd|jdddS(sqVerify consistent handling of empty-element tags, no matter how they come in through the markup. s


s


N(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyR<vscCs<d}|j|}|jd|jj|j|dS(s8Prevent recurrence of a bug in the html5lib treebuilder.s? foo N(RtassertNotEqualRROtbodyR (R tcontentR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt#test_head_tag_between_head_and_body}scCs)d}|j|}|j|jdS(s8Prevent recurrence of a bug in the html5lib treebuilder.s N(RR tarticle(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_multiple_copies_of_a_tags cCs}d}|j|}|j||j|j}|jd|jd|jd|jd|jd|jddS( sParsers don't need to *understand* namespaces, but at the very least they should not choke on namespaces or lose data.s4shttp://www.w3.org/1999/xhtmltxmlnss"http://www.w3.org/1998/Math/MathMLs xmlns:mathmlshttp://www.w3.org/2000/svgs xmlns:svgN(RRRWRO(R RRRO((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_basic_namespacess cCs6d}|j|}|jddg|jddS(NsREtbarR(RRR(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt-test_multivalued_attribute_value_becomes_listscCs/d}|j|}|jd|jjdS(NuDSacré bleu!u Sacré bleu!(RRRR}(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_can_parse_unicode_documentscCs;td}|jdd|}|j|jddS(s2Parsers should be able to work with SoupStrainers.R|s&A bold statementt parse_onlys boldN(RRRR(R tstrainerR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_soupstrainers   cCs|jdddS(Nss(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt7test_single_quote_attribute_values_become_double_quotess cCsd}|j|dS(Ns'a(R(R Rk((s//usr/lib/python2.7/site-packages/bs4/testing.pyt7test_attribute_values_with_nested_quotes_are_left_alonescCs?d}|j|}d|jd<|j|jjddS(Ns'asBrawls happen at "Bob's Bar"tattrs:a(RRERR(R RkR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt:test_attribute_values_with_double_nested_quotes_get_quoteds   cCs$|jdd|jdddS(Ns+s/s.foos2foo(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt.test_ampersand_in_attribute_value_gets_escapeds  cCs|jddS(Ns/(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt7test_escaped_ampersand_in_attribute_value_is_left_alonescCs d}d}|j||dS(Ns-

<<sacré bleu!>>

u#

<<sacré bleu!>>

(R(R Rktexpected((s//usr/lib/python2.7/site-packages/bs4/testing.pyt1test_entities_in_strings_converted_during_parsingscCs/d}|j|}|j|jjddS(Ns

Foo

u ‘Foo’(RRRJR}(R tquoteR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt)test_smart_quotes_converted_on_the_way_ins  cCs)|jd}|j|jjddS(Ns  u iu  (RRRR}(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyt0test_non_breaking_spaces_converted_on_the_way_inscCsDd}djd}|j|}|j|jjd|dS(Ns-

<<sacré bleu!>>

u#

<<sacré bleu!>>

sutf-8(RWRRRJ(R RkRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt&test_entities_converted_on_the_way_outscCshd}|jd}|j|}|jd}|jdd}|jd}|j||dS(Nu

Sacré bleu!

s iso-8859-1sutf-8s ISO-Latin-1(RWRRXR(R t unicode_htmltiso_latin_htmlRtresultR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_real_iso_latin_documentscCsld}|jd}|j|}|j|jd|jd|j|jd|jddS(Nsk
Shift-JISŃR[fBOꂽ{̃t@CłB
s shift-jissutf-8teuc_jp(RRRRW(R tshift_jis_htmlRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_real_shift_jis_documents "cCsJd}|j|dd}|j|jd|jdjddS(NsHebrew (ISO 8859-8) in Visual Directionality

Hebrew (ISO 8859-8) in Visual Directionality

t from_encodings iso8859-8sutf-8(RRRWR(R thebrew_documentR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_real_hebrew_document$s  cCsd}d|}|j|}|jdidd6}|d}|jd||jt|t|jd|jd dS( NsEsj %s Shift-JIS markup goes here.R0s Content-types http-equivRstext/html; charset=x-sjisstext/html; charset=utf8R\(RRiRRdt isinstanceRRW(R tmeta_tagRRt parsed_metaR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt'test_meta_tag_reflects_current_encoding1s cCsd}d|}|j|}|jddd}|d}|jd||jt|t|jd|jddS( Ns'sj %s Shift-JIS markup goes here.R0tidtencodingtcharsetsx-sjisR\(RRiRRdRRRW(R RRRRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt3test_html5_style_meta_tag_reflects_current_encodingMs cCs9|jd}d|jd<|jd|jjdS(Ns textRREstext(RRRR(R tdata((s//usr/lib/python2.7/site-packages/bs4/testing.pyt5test_tag_with_no_attributes_can_have_attributes_addedes (:R!R"t__doc__R<RDRNRFRPRRRSRTRURVRYR[R]R`RbRcReRfRjRoRuRyR~RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR(((s//usr/lib/python2.7/site-packages/bs4/testing.pyR$?sr                                          tXMLTreeBuilderSmokeTestcBseZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZRS(cCsc|jd}tj|d}tj|}|j|jt|j|j|jdS(Ns fooi(RR=R>R?RR@RR(R RARBRC((s//usr/lib/python2.7/site-packages/bs4/testing.pyRDls cCs)|jd}|j|jddS(Nss. (RRRW(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_docstring_generateduscCs2d}|j|}|j||jddS(Ns, R\(RRRW(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_xml_declarationzscCs2d}|j|}|j||jddS(Ns< R\(RRRW(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyR]scCs2d}|j|}|j|jd|dS(sGA real XHTML document should come out *exactly* the same as it went in.s Hello. Goodbye. sutf-8N(RRRW(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyRYscCs/d}|j|}|j||jdS(Ns (RRRW(R tdocR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_nested_namespacesscCsDd}t|d}d|j_|j}|jd|kdS(Ns/ slxml-xmlsconsole.log("< < hey > > ");s< < hey > >(RRgR}RWRd(R RRtencoded((s//usr/lib/python2.7/site-packages/bs4/testing.pyt5test_formatter_processes_script_tag_for_xml_documentss   cCs/d}|j|}|jd|jjdS(Nu?Sacré bleu!u Sacré bleu!(RRtrootR}(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyRscCs2d}|j|}|jt|j|dS(Nsb2012-07-02T20:33:42Zcd(RRtunicodetrss(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_popping_namespaced_tagscCs,|jd}|j|jdddS(Nstlatin1s/ (RRRW(R R((s//usr/lib/python2.7/site-packages/bs4/testing.pyt(test_docstring_includes_correct_encodings cCs>dddd}|j|}|j|jd|dS( s<A large XML document should come out the same as it went in.s- t0ii ssutf-8Ni(RRRW(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_large_xml_documentscCs!|jdd|jddS(Ns

s

s

foo

(R(R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyt9test_tags_are_empty_element_if_and_only_if_they_are_emptyscCsJd}|j|}|j}|jd|d|jd|ddS(NsThis tag is in the a namespaceThis tag is in the b namespaceshttp://example.com/sxmlns:ashttp://example.net/sxmlns:b(RRR(R RRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_namespaces_are_preserveds  cCs2d}|j|}|jt|j|dS(NsN

20010504

(RRRRJ(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_closing_namespaced_tagscCs2d}|j|}|jt|j|dS(Nss(RRRRE(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_namespaced_attributesscCs2d}|j|}|jt|j|dS(Nsbar(RRRRE(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt(test_namespaced_attributes_xml_namespacescCsd}|j|}|jdt|jd|jdt|jd|jdt|jd|jdt|jddd |jdt|jddgdS( Ns foo bar baz ittagisns1:tagisns2:tagtkeytvalue(RRRIRZ(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_find_by_prefixed_names %cCsGd}|j|}|j}tj|}|j|j|jdS(Nsf (RtdocumentR^Rtprefix(R txmlRRt duplicate((s//usr/lib/python2.7/site-packages/bs4/testing.pyt!test_copy_tag_preserves_namespaces  (R!R"RDRRR]RYRRRRRRRRRRRRR(((s//usr/lib/python2.7/site-packages/bs4/testing.pyRjs$            tHTML5TreeBuilderSmokeTestcBs;eZdZdZdZdZdZdZRS(s2Smoke test for a tree builder that supports HTML5.cCsdS(N((R ((s//usr/lib/python2.7/site-packages/bs4/testing.pyRYscCs/d}|j|}|jd|jjdS(Nsshttp://www.w3.org/1999/xhtml(RRRt namespace(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_html_tags_have_namespacescCsKd}|j|}d}|j||jj|j||jjdS(Nsshttp://www.w3.org/2000/svg(RRtsvgRtcircle(R RRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_svg_tags_have_namespace s cCsKd}|j|}d}|j||jj|j||jjdS(Ns5s"http://www.w3.org/1998/Math/MathML(RRtmathRtmsqrt(R RRR((s//usr/lib/python2.7/site-packages/bs4/testing.pyttest_mathml_tags_have_namespaces cCsjd}|j|}|jt|jdt|j|jdd|jd|jdjjdS(Ns3is$?xml version="1.0" encoding="utf-8"?RO(RRdRRGRRRR;(R RR((s//usr/lib/python2.7/site-packages/bs4/testing.pyt$test_xml_declaration_becomes_comments (R!R"RRYRRRR(((s//usr/lib/python2.7/site-packages/bs4/testing.pyRs     csdfd}|S(Nc_sdS(N(R(ttesttargsR((s//usr/lib/python2.7/site-packages/bs4/testing.pytnothing!scsr S|SdS(N((t test_item(t conditionR(s//usr/lib/python2.7/site-packages/bs4/testing.pyt decorator$s((RtreasonR((RRs//usr/lib/python2.7/site-packages/bs4/testing.pytskipIf s (Rt __license__R=R^t functoolstunittestRtbs4Rt bs4.elementRRRRRt bs4.builderRR R tobjectR$RRR(((s//usr/lib/python2.7/site-packages/bs4/testing.pyts"    (%-#PK{0] element.pyonu[ 6eb[c@sdZyddlmZWn#ek r?ZddlmZnXddlZddlZddlZddl Z ddl m Z dZ ej ddkZejdZd Zd efd YZd efd YZdefdYZdefdYZde fdYZdefdYZdefdYZdefdYZdefdYZdefdYZdefdYZd efd!YZd"eefd#YZ d$e fd%YZ!d&e!fd'YZ"d(e!fd)YZ#d*e#fd+YZ$d,e!fd-YZ%d.e!fd/YZ&d0e!fd1YZ'd2efd3YZ(d4efd5YZ)d6e*fd7YZ+dS(8tMITi(tCallableN(tEntitySubstitutionsutf-8iis\s+cs1tfd}|jfd}|S(s>Alias one attribute name to another for backward compatibilitycs t|S(N(tgetattr(tself(tattr(s//usr/lib/python2.7/site-packages/bs4/element.pytaliasscs t|S(N(tsetattr(R(R(s//usr/lib/python2.7/site-packages/bs4/element.pyRs(tpropertytsetter(RR((Rs//usr/lib/python2.7/site-packages/bs4/element.pyt_aliasstNamespacedAttributecBseZddZRS(cCs{|dkr!tj||}n;|dkrBtj||}ntj||d|}||_||_||_|S(Nt:(tNonetunicodet__new__tprefixtnamet namespace(tclsRRRtobj((s//usr/lib/python2.7/site-packages/bs4/element.pyR"s     N(t__name__t __module__R R(((s//usr/lib/python2.7/site-packages/bs4/element.pyR st%AttributeValueWithCharsetSubstitutioncBseZdZRS(s=A stand-in object for a character encoding specified in HTML.(RRt__doc__(((s//usr/lib/python2.7/site-packages/bs4/element.pyR/stCharsetMetaAttributeValuecBs eZdZdZdZRS(sA generic stand-in for the value of a meta tag's 'charset' attribute. When Beautiful Soup parses the markup '', the value of the 'charset' attribute will be one of these objects. cCstj||}||_|S(N(RRtoriginal_value(RRR((s//usr/lib/python2.7/site-packages/bs4/element.pyR9s cCs|S(N((Rtencoding((s//usr/lib/python2.7/site-packages/bs4/element.pytencode>s(RRRRR(((s//usr/lib/python2.7/site-packages/bs4/element.pyR2s tContentMetaAttributeValuecBs5eZdZejdejZdZdZRS(sA generic stand-in for the value of a meta tag's 'content' attribute. When Beautiful Soup parses the markup: The value of the 'content' attribute will be one of these objects. s((^|;)\s*charset=)([^;]*)cCsM|jj|}|dkr.tjt|Stj||}||_|S(N(t CHARSET_REtsearchR RRR(RRtmatchR((s//usr/lib/python2.7/site-packages/bs4/element.pyRMs   cs%fd}|jj||jS(Ncs|jdS(Ni(tgroup(R (R(s//usr/lib/python2.7/site-packages/bs4/element.pytrewriteXs(RtsubR(RRR"((Rs//usr/lib/python2.7/site-packages/bs4/element.pyRWs( RRRtretcompiletMRRR(((s//usr/lib/python2.7/site-packages/bs4/element.pyRBs tHTMLAwareEntitySubstitutioncBsneZdZeddgZedgZeddgZedZedZ edZ RS(s%Entity substitution rules that are aware of some HTML quirks. Specifically, the contents of Hello, world! ''' soup = self.soup(html) self.assertEqual("text/javascript", soup.find('script')['type']) def test_comment(self): # Comments are represented as Comment objects. markup = "

foobaz

" self.assertSoupEquals(markup) soup = self.soup(markup) comment = soup.find(text="foobar") self.assertEqual(comment.__class__, Comment) # The comment is properly integrated into the tree. foo = soup.find(text="foo") self.assertEqual(comment, foo.next_element) baz = soup.find(text="baz") self.assertEqual(comment, baz.previous_element) def test_preserved_whitespace_in_pre_and_textarea(self): """Whitespace must be preserved in
 and "
        self.assertSoupEquals(pre_markup)
        self.assertSoupEquals(textarea_markup)

        soup = self.soup(pre_markup)
        self.assertEqual(soup.pre.prettify(), pre_markup)

        soup = self.soup(textarea_markup)
        self.assertEqual(soup.textarea.prettify(), textarea_markup)

        soup = self.soup("")
        self.assertEqual(soup.textarea.prettify(), "")

    def test_nested_inline_elements(self):
        """Inline elements can be nested indefinitely."""
        b_tag = "Inside a B tag"
        self.assertSoupEquals(b_tag)

        nested_b_tag = "

A nested tag

" self.assertSoupEquals(nested_b_tag) double_nested_b_tag = "

A doubly nested tag

" self.assertSoupEquals(nested_b_tag) def test_nested_block_level_elements(self): """Block elements can be nested.""" soup = self.soup('

Foo

') blockquote = soup.blockquote self.assertEqual(blockquote.p.b.string, 'Foo') self.assertEqual(blockquote.b.string, 'Foo') def test_correctly_nested_tables(self): """One table can go inside another one.""" markup = ('' '' "') self.assertSoupEquals( markup, '
Here's another table:" '' '' '
foo
Here\'s another table:' '
foo
' '
') self.assertSoupEquals( "" "" "
Foo
Bar
Baz
") def test_deeply_nested_multivalued_attribute(self): # html5lib can set the attributes of the same tag many times # as it rearranges the tree. This has caused problems with # multivalued attributes. markup = '
' soup = self.soup(markup) self.assertEqual(["css"], soup.div.div['class']) def test_multivalued_attribute_on_html(self): # html5lib uses a different API to set the attributes ot the # tag. This has caused problems with multivalued # attributes. markup = '' soup = self.soup(markup) self.assertEqual(["a", "b"], soup.html['class']) def test_angle_brackets_in_attribute_values_are_escaped(self): self.assertSoupEquals('', '') def test_strings_resembling_character_entity_references(self): # "&T" and "&p" look like incomplete character entities, but they are # not. self.assertSoupEquals( u"

• AT&T is in the s&p 500

", u"

\u2022 AT&T is in the s&p 500

" ) def test_entities_in_foreign_document_encoding(self): # “ and ” are invalid numeric entities referencing # Windows-1252 characters. - references a character common # to Windows-1252 and Unicode, and ☃ references a # character only found in Unicode. # # All of these entities should be converted to Unicode # characters. markup = "

“Hello” -☃

" soup = self.soup(markup) self.assertEquals(u"“Hello” -☃", soup.p.string) def test_entities_in_attributes_converted_to_unicode(self): expect = u'

' self.assertSoupEquals('

', expect) self.assertSoupEquals('

', expect) self.assertSoupEquals('

', expect) self.assertSoupEquals('

', expect) def test_entities_in_text_converted_to_unicode(self): expect = u'

pi\N{LATIN SMALL LETTER N WITH TILDE}ata

' self.assertSoupEquals("

piñata

", expect) self.assertSoupEquals("

piñata

", expect) self.assertSoupEquals("

piñata

", expect) self.assertSoupEquals("

piñata

", expect) def test_quot_entity_converted_to_quotation_mark(self): self.assertSoupEquals("

I said "good day!"

", '

I said "good day!"

') def test_out_of_range_entity(self): expect = u"\N{REPLACEMENT CHARACTER}" self.assertSoupEquals("�", expect) self.assertSoupEquals("�", expect) self.assertSoupEquals("�", expect) def test_multipart_strings(self): "Mostly to prevent a recurrence of a bug in the html5lib treebuilder." soup = self.soup("

\nfoo

") self.assertEqual("p", soup.h2.string.next_element.name) self.assertEqual("p", soup.p.name) self.assertConnectedness(soup) def test_empty_element_tags(self): """Verify consistent handling of empty-element tags, no matter how they come in through the markup. """ self.assertSoupEquals('


', "


") self.assertSoupEquals('


', "


") def test_head_tag_between_head_and_body(self): "Prevent recurrence of a bug in the html5lib treebuilder." content = """ foo """ soup = self.soup(content) self.assertNotEqual(None, soup.html.body) self.assertConnectedness(soup) def test_multiple_copies_of_a_tag(self): "Prevent recurrence of a bug in the html5lib treebuilder." content = """ """ soup = self.soup(content) self.assertConnectedness(soup.article) def test_basic_namespaces(self): """Parsers don't need to *understand* namespaces, but at the very least they should not choke on namespaces or lose data.""" markup = b'4' soup = self.soup(markup) self.assertEqual(markup, soup.encode()) html = soup.html self.assertEqual('http://www.w3.org/1999/xhtml', soup.html['xmlns']) self.assertEqual( 'http://www.w3.org/1998/Math/MathML', soup.html['xmlns:mathml']) self.assertEqual( 'http://www.w3.org/2000/svg', soup.html['xmlns:svg']) def test_multivalued_attribute_value_becomes_list(self): markup = b'' soup = self.soup(markup) self.assertEqual(['foo', 'bar'], soup.a['class']) # # Generally speaking, tests below this point are more tests of # Beautiful Soup than tests of the tree builders. But parsers are # weird, so we run these tests separately for every tree builder # to detect any differences between them. # def test_can_parse_unicode_document(self): # A seemingly innocuous document... but it's in Unicode! And # it contains characters that can't be represented in the # encoding found in the declaration! The horror! markup = u'Sacr\N{LATIN SMALL LETTER E WITH ACUTE} bleu!' soup = self.soup(markup) self.assertEqual(u'Sacr\xe9 bleu!', soup.body.string) def test_soupstrainer(self): """Parsers should be able to work with SoupStrainers.""" strainer = SoupStrainer("b") soup = self.soup("A bold statement", parse_only=strainer) self.assertEqual(soup.decode(), "bold") def test_single_quote_attribute_values_become_double_quotes(self): self.assertSoupEquals("", '') def test_attribute_values_with_nested_quotes_are_left_alone(self): text = """a""" self.assertSoupEquals(text) def test_attribute_values_with_double_nested_quotes_get_quoted(self): text = """a""" soup = self.soup(text) soup.foo['attr'] = 'Brawls happen at "Bob\'s Bar"' self.assertSoupEquals( soup.foo.decode(), """a""") def test_ampersand_in_attribute_value_gets_escaped(self): self.assertSoupEquals('', '') self.assertSoupEquals( 'foo', 'foo') def test_escaped_ampersand_in_attribute_value_is_left_alone(self): self.assertSoupEquals('') def test_entities_in_strings_converted_during_parsing(self): # Both XML and HTML entities are converted to Unicode characters # during parsing. text = "

<<sacré bleu!>>

" expected = u"

<<sacr\N{LATIN SMALL LETTER E WITH ACUTE} bleu!>>

" self.assertSoupEquals(text, expected) def test_smart_quotes_converted_on_the_way_in(self): # Microsoft smart quotes are converted to Unicode characters during # parsing. quote = b"

\x91Foo\x92

" soup = self.soup(quote) self.assertEqual( soup.p.string, u"\N{LEFT SINGLE QUOTATION MARK}Foo\N{RIGHT SINGLE QUOTATION MARK}") def test_non_breaking_spaces_converted_on_the_way_in(self): soup = self.soup("  ") self.assertEqual(soup.a.string, u"\N{NO-BREAK SPACE}" * 2) def test_entities_converted_on_the_way_out(self): text = "

<<sacré bleu!>>

" expected = u"

<<sacr\N{LATIN SMALL LETTER E WITH ACUTE} bleu!>>

".encode("utf-8") soup = self.soup(text) self.assertEqual(soup.p.encode("utf-8"), expected) def test_real_iso_latin_document(self): # Smoke test of interrelated functionality, using an # easy-to-understand document. # Here it is in Unicode. Note that it claims to be in ISO-Latin-1. unicode_html = u'

Sacr\N{LATIN SMALL LETTER E WITH ACUTE} bleu!

' # That's because we're going to encode it into ISO-Latin-1, and use # that to test. iso_latin_html = unicode_html.encode("iso-8859-1") # Parse the ISO-Latin-1 HTML. soup = self.soup(iso_latin_html) # Encode it to UTF-8. result = soup.encode("utf-8") # What do we expect the result to look like? Well, it would # look like unicode_html, except that the META tag would say # UTF-8 instead of ISO-Latin-1. expected = unicode_html.replace("ISO-Latin-1", "utf-8") # And, of course, it would be in UTF-8, not Unicode. expected = expected.encode("utf-8") # Ta-da! self.assertEqual(result, expected) def test_real_shift_jis_document(self): # Smoke test to make sure the parser can handle a document in # Shift-JIS encoding, without choking. shift_jis_html = ( b'
'
            b'\x82\xb1\x82\xea\x82\xcdShift-JIS\x82\xc5\x83R\x81[\x83f'
            b'\x83B\x83\x93\x83O\x82\xb3\x82\xea\x82\xbd\x93\xfa\x96{\x8c'
            b'\xea\x82\xcc\x83t\x83@\x83C\x83\x8b\x82\xc5\x82\xb7\x81B'
            b'
') unicode_html = shift_jis_html.decode("shift-jis") soup = self.soup(unicode_html) # Make sure the parse tree is correctly encoded to various # encodings. self.assertEqual(soup.encode("utf-8"), unicode_html.encode("utf-8")) self.assertEqual(soup.encode("euc_jp"), unicode_html.encode("euc_jp")) def test_real_hebrew_document(self): # A real-world test to make sure we can convert ISO-8859-9 (a # Hebrew encoding) to UTF-8. hebrew_document = b'Hebrew (ISO 8859-8) in Visual Directionality

Hebrew (ISO 8859-8) in Visual Directionality

\xed\xe5\xec\xf9' soup = self.soup( hebrew_document, from_encoding="iso8859-8") # Some tree builders call it iso8859-8, others call it iso-8859-9. # That's not a difference we really care about. assert soup.original_encoding in ('iso8859-8', 'iso-8859-8') self.assertEqual( soup.encode('utf-8'), hebrew_document.decode("iso8859-8").encode("utf-8")) def test_meta_tag_reflects_current_encoding(self): # Here's the tag saying that a document is # encoded in Shift-JIS. meta_tag = ('') # Here's a document incorporating that meta tag. shift_jis_html = ( '\n%s\n' '' 'Shift-JIS markup goes here.') % meta_tag soup = self.soup(shift_jis_html) # Parse the document, and the charset is seemingly unaffected. parsed_meta = soup.find('meta', {'http-equiv': 'Content-type'}) content = parsed_meta['content'] self.assertEqual('text/html; charset=x-sjis', content) # But that value is actually a ContentMetaAttributeValue object. self.assertTrue(isinstance(content, ContentMetaAttributeValue)) # And it will take on a value that reflects its current # encoding. self.assertEqual('text/html; charset=utf8', content.encode("utf8")) # For the rest of the story, see TestSubstitutions in # test_tree.py. def test_html5_style_meta_tag_reflects_current_encoding(self): # Here's the tag saying that a document is # encoded in Shift-JIS. meta_tag = ('') # Here's a document incorporating that meta tag. shift_jis_html = ( '\n%s\n' '' 'Shift-JIS markup goes here.') % meta_tag soup = self.soup(shift_jis_html) # Parse the document, and the charset is seemingly unaffected. parsed_meta = soup.find('meta', id="encoding") charset = parsed_meta['charset'] self.assertEqual('x-sjis', charset) # But that value is actually a CharsetMetaAttributeValue object. self.assertTrue(isinstance(charset, CharsetMetaAttributeValue)) # And it will take on a value that reflects its current # encoding. self.assertEqual('utf8', charset.encode("utf8")) def test_tag_with_no_attributes_can_have_attributes_added(self): data = self.soup("text") data.a['foo'] = 'bar' self.assertEqual('text', data.a.decode()) class XMLTreeBuilderSmokeTest(object): def test_pickle_and_unpickle_identity(self): # Pickling a tree, then unpickling it, yields a tree identical # to the original. tree = self.soup("foo") dumped = pickle.dumps(tree, 2) loaded = pickle.loads(dumped) self.assertEqual(loaded.__class__, BeautifulSoup) self.assertEqual(loaded.decode(), tree.decode()) def test_docstring_generated(self): soup = self.soup("") self.assertEqual( soup.encode(), b'\n') def test_xml_declaration(self): markup = b"""\n""" soup = self.soup(markup) self.assertEqual(markup, soup.encode("utf8")) def test_processing_instruction(self): markup = b"""\n""" soup = self.soup(markup) self.assertEqual(markup, soup.encode("utf8")) def test_real_xhtml_document(self): """A real XHTML document should come out *exactly* the same as it went in.""" markup = b""" Hello. Goodbye. """ soup = self.soup(markup) self.assertEqual( soup.encode("utf-8"), markup) def test_nested_namespaces(self): doc = b""" """ soup = self.soup(doc) self.assertEqual(doc, soup.encode()) def test_formatter_processes_script_tag_for_xml_documents(self): doc = """ """ soup = BeautifulSoup(doc, "lxml-xml") # lxml would have stripped this while parsing, but we can add # it later. soup.script.string = 'console.log("< < hey > > ");' encoded = soup.encode() self.assertTrue(b"< < hey > >" in encoded) def test_can_parse_unicode_document(self): markup = u'Sacr\N{LATIN SMALL LETTER E WITH ACUTE} bleu!' soup = self.soup(markup) self.assertEqual(u'Sacr\xe9 bleu!', soup.root.string) def test_popping_namespaced_tag(self): markup = 'b2012-07-02T20:33:42Zcd' soup = self.soup(markup) self.assertEqual( unicode(soup.rss), markup) def test_docstring_includes_correct_encoding(self): soup = self.soup("") self.assertEqual( soup.encode("latin1"), b'\n') def test_large_xml_document(self): """A large XML document should come out the same as it went in.""" markup = (b'\n' + b'0' * (2**12) + b'') soup = self.soup(markup) self.assertEqual(soup.encode("utf-8"), markup) def test_tags_are_empty_element_if_and_only_if_they_are_empty(self): self.assertSoupEquals("

", "

") self.assertSoupEquals("

foo

") def test_namespaces_are_preserved(self): markup = 'This tag is in the a namespaceThis tag is in the b namespace' soup = self.soup(markup) root = soup.root self.assertEqual("http://example.com/", root['xmlns:a']) self.assertEqual("http://example.net/", root['xmlns:b']) def test_closing_namespaced_tag(self): markup = '

20010504

' soup = self.soup(markup) self.assertEqual(unicode(soup.p), markup) def test_namespaced_attributes(self): markup = '' soup = self.soup(markup) self.assertEqual(unicode(soup.foo), markup) def test_namespaced_attributes_xml_namespace(self): markup = 'bar' soup = self.soup(markup) self.assertEqual(unicode(soup.foo), markup) def test_find_by_prefixed_name(self): doc = """ foo bar baz """ soup = self.soup(doc) # There are three tags. self.assertEqual(3, len(soup.find_all('tag'))) # But two of them are ns1:tag and one of them is ns2:tag. self.assertEqual(2, len(soup.find_all('ns1:tag'))) self.assertEqual(1, len(soup.find_all('ns2:tag'))) self.assertEqual(1, len(soup.find_all('ns2:tag', key='value'))) self.assertEqual(3, len(soup.find_all(['ns1:tag', 'ns2:tag']))) def test_copy_tag_preserves_namespace(self): xml = """ """ soup = self.soup(xml) tag = soup.document duplicate = copy.copy(tag) # The two tags have the same namespace prefix. self.assertEqual(tag.prefix, duplicate.prefix) class HTML5TreeBuilderSmokeTest(HTMLTreeBuilderSmokeTest): """Smoke test for a tree builder that supports HTML5.""" def test_real_xhtml_document(self): # Since XHTML is not HTML5, HTML5 parsers are not tested to handle # XHTML documents in any particular way. pass def test_html_tags_have_namespace(self): markup = "" soup = self.soup(markup) self.assertEqual("http://www.w3.org/1999/xhtml", soup.a.namespace) def test_svg_tags_have_namespace(self): markup = '' soup = self.soup(markup) namespace = "http://www.w3.org/2000/svg" self.assertEqual(namespace, soup.svg.namespace) self.assertEqual(namespace, soup.circle.namespace) def test_mathml_tags_have_namespace(self): markup = '5' soup = self.soup(markup) namespace = 'http://www.w3.org/1998/Math/MathML' self.assertEqual(namespace, soup.math.namespace) self.assertEqual(namespace, soup.msqrt.namespace) def test_xml_declaration_becomes_comment(self): markup = '' soup = self.soup(markup) self.assertTrue(isinstance(soup.contents[0], Comment)) self.assertEqual(soup.contents[0], '?xml version="1.0" encoding="utf-8"?') self.assertEqual("html", soup.contents[0].next_element.name) def skipIf(condition, reason): def nothing(test, *args, **kwargs): return None def decorator(test_item): if condition: return nothing else: return test_item return decorator PK{0]&-tests/test_lxml.pycnu[ \[c@sKdZddlZddlZy"ddlZeZejjZWne k raZ e ZdZnXerddl m Z mZnddlmZmZddlmZmZmZddlmZddlmZdd lmZmZmZmZee d d eefd YZee d deefdYZdS(s@Tests to ensure that the lxml tree builder generates good trees.iNi(tLXMLTreeBuildertLXMLTreeBuilderForXML(t BeautifulSouptBeautifulStoneSoup(tCommenttDoctypet SoupStrainer(tskipIf(ttest_htmlparser(tHTMLTreeBuilderSmokeTesttXMLTreeBuilderSmokeTesttSoupTestRs;lxml seems not to be present, not testing its tree builder.tLXMLTreeBuilderSmokeTestcBs]eZdZedZdZdZee p@e d kdd Z d Z RS( s!See ``HTMLTreeBuilderSmokeTest``.cCstS(N(R(tself((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pytdefault_builder%scCs4|jdd|jdd|jdddS(Ns

foo�bar

s

foobar

s

foo�bar

s

foo�bar

(tassertSoupEquals(R ((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyttest_out_of_range_entity)s   cCsdS(N((R ((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyt*test_entities_in_foreign_document_encoding1siiiis@Skipping doctype test for old version of lxml to avoid segfault.cCs6|jd}|jd}|jd|jdS(Ns it(tsouptcontentst assertEqualtstrip(R Rtdoctype((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyttest_empty_doctype:s cCsdtjdt}td}WdQX|jdt|j|jdt|dj kdS(Ntrecordsus&BeautifulStoneSoup class is deprecatedi( twarningstcatch_warningstTrueRRtunicodetbt assertTruetstrtmessage(R twR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyt%test_beautifulstonesoup_is_xml_parserBs(iiii( t__name__t __module__t__doc__tpropertyRRRRt LXML_PRESENTt LXML_VERSIONRR#(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyR s  s?lxml seems not to be present, not testing its XML tree builder.tLXMLXMLTreeBuilderSmokeTestcBseZdZedZRS(s!See ``HTMLTreeBuilderSmokeTest``.cCstS(N(R(R ((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyRPs(R$R%R&R'R(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyR*Js(i(R&treRt lxml.etreetlxmlRR(tetreeR)t ImportErrortetFalset bs4.builderRRtbs4RRt bs4.elementRRRt bs4.testingRt bs4.testsRR R R R R*(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_lxml.pyts0    " ( PK{0]t tests/test_htmlparser.pyonu[ Y=K[c@sdZddlmZddlZddlmZmZddlmZddl m Z deefdYZ d efd YZ dS( sGTests to ensure that the html.parser tree builder generates good trees.i(t set_traceN(tSoupTesttHTMLTreeBuilderSmokeTest(tHTMLParserTreeBuilder(tBeautifulSoupHTMLParsertHTMLParserTreeBuilderSmokeTestcBsDeZedZdZdZdZdZdZRS(cCstS(N(R(tself((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pytdefault_builder scCsdS(N((R((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyttest_namespaced_system_doctypescCsdS(N((R((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyttest_namespaced_public_doctypescCsV|jd}tj|d}tj|}|jt|jt|jdS(sfUnlike most tree builders, HTMLParserTreeBuilder and will be restored after pickling. s
fooiN(tsouptpickletdumpstloadst assertTruet isinstancetbuilderttype(Rttreetdumpedtloaded((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyttest_builder_is_pickledscCs$|jdd|jdddS(Ns





s


s


t(tassertSoupEquals(R((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyt)test_redundant_empty_element_closing_tags!scCs|jdddS(Ns foo &# barsfoo &# bar(R(R((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyttest_empty_element%s( t__name__t __module__tpropertyRRR RRR(((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyR s    tTestHTMLParserSubclasscBseZdZRS(cCst}|jddS(slVerify that our HTMLParser subclass implements error() in a way that doesn't cause a crash. s don't crashN(Rterror(Rtparser((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyt test_error,s (RRR (((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyR+s( t__doc__tpdbRR t bs4.testingRRt bs4.builderRtbs4.builder._htmlparserRRR(((s=/usr/lib/python2.7/site-packages/bs4/tests/test_htmlparser.pyts !PK{0]jtests/test_builder_registry.pynu["""Tests of the builder registry.""" import unittest import warnings from bs4 import BeautifulSoup from bs4.builder import ( builder_registry as registry, HTMLParserTreeBuilder, TreeBuilderRegistry, ) try: from bs4.builder import HTML5TreeBuilder HTML5LIB_PRESENT = True except ImportError: HTML5LIB_PRESENT = False try: from bs4.builder import ( LXMLTreeBuilderForXML, LXMLTreeBuilder, ) LXML_PRESENT = True except ImportError: LXML_PRESENT = False class BuiltInRegistryTest(unittest.TestCase): """Test the built-in registry with the default builders registered.""" def test_combination(self): if LXML_PRESENT: self.assertEqual(registry.lookup('fast', 'html'), LXMLTreeBuilder) if LXML_PRESENT: self.assertEqual(registry.lookup('permissive', 'xml'), LXMLTreeBuilderForXML) self.assertEqual(registry.lookup('strict', 'html'), HTMLParserTreeBuilder) if HTML5LIB_PRESENT: self.assertEqual(registry.lookup('html5lib', 'html'), HTML5TreeBuilder) def test_lookup_by_markup_type(self): if LXML_PRESENT: self.assertEqual(registry.lookup('html'), LXMLTreeBuilder) self.assertEqual(registry.lookup('xml'), LXMLTreeBuilderForXML) else: self.assertEqual(registry.lookup('xml'), None) if HTML5LIB_PRESENT: self.assertEqual(registry.lookup('html'), HTML5TreeBuilder) else: self.assertEqual(registry.lookup('html'), HTMLParserTreeBuilder) def test_named_library(self): if LXML_PRESENT: self.assertEqual(registry.lookup('lxml', 'xml'), LXMLTreeBuilderForXML) self.assertEqual(registry.lookup('lxml', 'html'), LXMLTreeBuilder) if HTML5LIB_PRESENT: self.assertEqual(registry.lookup('html5lib'), HTML5TreeBuilder) self.assertEqual(registry.lookup('html.parser'), HTMLParserTreeBuilder) def test_beautifulsoup_constructor_does_lookup(self): with warnings.catch_warnings(record=True) as w: # This will create a warning about not explicitly # specifying a parser, but we'll ignore it. # You can pass in a string. BeautifulSoup("", features="html") # Or a list of strings. BeautifulSoup("", features=["html", "fast"]) # You'll get an exception if BS can't find an appropriate # builder. self.assertRaises(ValueError, BeautifulSoup, "", features="no-such-feature") class RegistryTest(unittest.TestCase): """Test the TreeBuilderRegistry class in general.""" def setUp(self): self.registry = TreeBuilderRegistry() def builder_for_features(self, *feature_list): cls = type('Builder_' + '_'.join(feature_list), (object,), {'features' : feature_list}) self.registry.register(cls) return cls def test_register_with_no_features(self): builder = self.builder_for_features() # Since the builder advertises no features, you can't find it # by looking up features. self.assertEqual(self.registry.lookup('foo'), None) # But you can find it by doing a lookup with no features, if # this happens to be the only registered builder. self.assertEqual(self.registry.lookup(), builder) def test_register_with_features_makes_lookup_succeed(self): builder = self.builder_for_features('foo', 'bar') self.assertEqual(self.registry.lookup('foo'), builder) self.assertEqual(self.registry.lookup('bar'), builder) def test_lookup_fails_when_no_builder_implements_feature(self): builder = self.builder_for_features('foo', 'bar') self.assertEqual(self.registry.lookup('baz'), None) def test_lookup_gets_most_recent_registration_when_no_feature_specified(self): builder1 = self.builder_for_features('foo') builder2 = self.builder_for_features('bar') self.assertEqual(self.registry.lookup(), builder2) def test_lookup_fails_when_no_tree_builders_registered(self): self.assertEqual(self.registry.lookup(), None) def test_lookup_gets_most_recent_builder_supporting_all_features(self): has_one = self.builder_for_features('foo') has_the_other = self.builder_for_features('bar') has_both_early = self.builder_for_features('foo', 'bar', 'baz') has_both_late = self.builder_for_features('foo', 'bar', 'quux') lacks_one = self.builder_for_features('bar') has_the_other = self.builder_for_features('foo') # There are two builders featuring 'foo' and 'bar', but # the one that also features 'quux' was registered later. self.assertEqual(self.registry.lookup('foo', 'bar'), has_both_late) # There is only one builder featuring 'foo', 'bar', and 'baz'. self.assertEqual(self.registry.lookup('foo', 'bar', 'baz'), has_both_early) def test_lookup_fails_when_cannot_reconcile_requested_features(self): builder1 = self.builder_for_features('foo', 'bar') builder2 = self.builder_for_features('foo', 'baz') self.assertEqual(self.registry.lookup('bar', 'baz'), None) PK{0]H9tests/test_htmlparser.pynu["""Tests to ensure that the html.parser tree builder generates good trees.""" from pdb import set_trace import pickle from bs4.testing import SoupTest, HTMLTreeBuilderSmokeTest from bs4.builder import HTMLParserTreeBuilder from bs4.builder._htmlparser import BeautifulSoupHTMLParser class HTMLParserTreeBuilderSmokeTest(SoupTest, HTMLTreeBuilderSmokeTest): @property def default_builder(self): return HTMLParserTreeBuilder() def test_namespaced_system_doctype(self): # html.parser can't handle namespaced doctypes, so skip this one. pass def test_namespaced_public_doctype(self): # html.parser can't handle namespaced doctypes, so skip this one. pass def test_builder_is_pickled(self): """Unlike most tree builders, HTMLParserTreeBuilder and will be restored after pickling. """ tree = self.soup("foo") dumped = pickle.dumps(tree, 2) loaded = pickle.loads(dumped) self.assertTrue(isinstance(loaded.builder, type(tree.builder))) def test_redundant_empty_element_closing_tags(self): self.assertSoupEquals('





', "


") self.assertSoupEquals('


', "") def test_empty_element(self): # This verifies that any buffered data present when the parser # finishes working is handled. self.assertSoupEquals("foo &# bar", "foo &# bar") class TestHTMLParserSubclass(SoupTest): def test_error(self): """Verify that our HTMLParser subclass implements error() in a way that doesn't cause a crash. """ parser = BeautifulSoupHTMLParser() parser.error("don't crash") PK{0]Stests/test_tree.pyonu[ _[c@sdZddlmZddlZddlZddlZddlZddlmZddl m Z m Z ddl m Z mZmZmZmZmZmZmZddlmZmZe jdek Ze jd ek Zd efd YZd efd YZdefdYZdefdYZdefdYZ defdYZ!defdYZ"defdYZ#defdYZ$de$fdYZ%de$fdYZ&d efd!YZ'd"e'fd#YZ(d$e'fd%YZ)d&efd'YZ*d(efd)YZ+d*efd+YZ,d,efd-YZ-d.efd/YZ.d0efd1YZ/d2efd3YZ0d4efd5YZ1d6efd7YZ2dS(8s8Tests for Beautiful Soup's tree traversal methods. The tree traversal methods are the main advantage of using Beautiful Soup over just using a parser. Different parsers will build different Beautiful Soup trees given the same markup, but all Beautiful Soup trees can be traversed with the methods tested here. i(t set_traceN(t BeautifulSoup(tbuilder_registrytHTMLParserTreeBuilder(tPY3KtCDatatCommentt DeclarationtDoctypetNavigableStringt SoupStrainertTag(tSoupTesttskipIftxmltlxmltTreeTestcBseZdZdZRS(cCs*|jg|D]}|j^q |dS(sMake sure that the given tags have the correct text. This is used in tests that define a bunch of tags, each containing a single string, and then select certain strings by some mechanism. N(t assertEqualtstring(tselfttagst should_matchttag((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt assertSelects+scCs+|jg|D]}|d^q |dS(sMake sure that the given tags have the correct IDs. This is used in tests that define a bunch of tags, each containing a single string, and then select certain strings by some mechanism. tidN(R(RRRR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pytassertSelectsIDs4s(t__name__t __module__RR(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyR)s tTestFindcBs;eZdZdZdZdZdZdZRS(sBasic tests of the find() method. find() just calls find_all() with limit=1, so it's not tested all that thouroughly here. cCs/|jd}|j|jdjddS(Ns 1234tbt2(tsoupRtfindR(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt test_find_tagEscCs/|jd}|j|jddddS(Nu

Räksmörgås

Ru Räksmörgås(RRR (RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_unicode_text_findIscCs<|jd}t||jd|jddjdS(Nu&

here it is

s here it isRu Räksmörgås(RtstrRR ttext(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_unicode_attribute_findMs cCs/|jd}|jdt|jdS(s)Test an optimization that finds all tags.sfoobariN(RRtlentfind_all(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_everythingSscCs2|jd}|jdt|jddS(s;Test an optimization that finds all tags with a given name.sfoobarbazitaN(RRR&R'(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_everything_with_nameXs(RRt__doc__R!R"R%R(R*(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyR>s     t TestFindAllcBs;eZdZdZdZdZdZdZRS(s%Basic tests of the find_all() method.cCs|jd}|j|jdddg|j|jdddg|j|jdddgddg|j|jdtjdddd g|j|jdtddd gd S( s'You can search the tree for text nodes.sFoobarRtbarubarR$tFoouFoos.*u»N(RRR'tretcompiletTrue(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_text_nodes`s"cCs|jd}|j|jddddddg|j|jddddg|j|jddd dddd d g|j|jddd dddd d gd S(s7You can limit the number of items returned by find_all.s(12345R)tlimitit1Rt3ii t4t5iN(RRR'(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_limitps("(cCsQ|jd}|j|ddddg|j|jdddgdS( Ns!123R)R3iR4RtfooR5(RRR(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt%test_calling_a_tag_is_calling_findall|scCs?|jd}g}|j||jg|j|dS(Ns(RtappendRR'(RRtl((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pytTtest_find_all_with_self_referential_data_structure_does_not_cause_infinite_recursions cCs|jd}|jd}|jt|d|jt}|jt|d|jdd}|jt|ddS(s%All find_all calls return a ResultSetsR)tsourceR$R9N(RR't assertTruethasattrR1(RRtresult((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_resultsets(RRR+R2R8R:R=RB(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyR,]s    tTestFindAllBasicNamespacescBseZdZRS(cCsU|jd}|jd|jdj|jd|jdidd6jdS(Ns04R6s mathml:msqrtR)tattrstredssvg:fill(RRR Rtname(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_by_namespaced_names(RRRG(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyRCstTestFindAllByNamecBsqeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z RS( s&Test ways of finding tags by tag name.cCs)tt|j|jd|_dS(NsFirst tag. Second tag. Third Nested tag. tag.(tsuperRtsetUpRttree(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyRJscCs&|j|jjdddgdS(NR)s First tag.s Nested tag.(RRKR'(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_tag_namescCs|j|jjddddg|j|jjddtddg|j|jjddtjdddgdS(NR)R$s First tag.s Nested tag.R(RRKR'R1R/R0(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_name_and_texts"cCs&|j|jjjddgdS(NR)s Nested tag.(RRKtcR'(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt!test_find_all_on_non_root_elementscCs#|j|jdddgdS(NR)s First tag.s Nested tag.(RRK(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt%test_calling_element_invokes_find_allscCs,|j|jjtdddgdS(NR)s First tag.s Nested tag.(RRKR'R (R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_tag_strainerscCs/|j|jjddgdddgdS(NR)Rs First tag.s Second tag.s Nested tag.(RRKR'(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_tag_namesscCs7|j|jjitd6td6dddgdS(NR)Rs First tag.s Second tag.s Nested tag.(RRKR'R1(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_tag_dictscCs2|j|jjtjddddgdS(Ns^[ab]$s First tag.s Second tag.s Nested tag.(RRKR'R/R0(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_tag_rescCs;d}|jd}|j|j|ddgdS(NcSs|j|jdkS(NR(RFtget(R((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pytid_matches_namessMatch 1. Does not match. Match 2.sMatch 1.sMatch 2.(RRR'(RRVRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt'test_find_all_with_tags_matching_methods   cCs|jd}|jdd}|jdtjd}|jdddg\}}|jd|j|jd|j|jd|j|jd|jdS(NsH
1
2
3
tdivsa dsa bR5R4(RR R/R0R'RR(RRtr1tr2tr3tr4((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt%test_find_with_multi_valued_attributes (RRR+RJRLRMRORPRQRRRSRTRWR](((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyRHs         tTestFindAllByAttributecBseZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZRS(cCs5|jd}|j|jddddgdS(Ns Matching a. Non-matching Matching b.a. Rtfirsts Matching a.s Matching b.(RRR'(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_attribute_names cCsdjd}djd}|j|}|j|jg|jd||j|jg|jd|jd|j|jg|jd|dgdS(Nuםולשtutf8uttitlessomething else(tencodeRRR)R'tdecode(RtpeacetdataR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt%test_find_all_by_utf8_attribute_values "+cCs~|jd}|j|jdddg|j|jdidd6dg|j|jdidd6d gdS( Ns0 Name match. Class match. Non-match. A tag called 'name1'. RFtname1sA tag called 'name1'.RDs Name match.tclass2tclasss Class match.(RRR'(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_attribute_dicts   cCs|jd}|j|jddddg|j|jddddg|j|jddd dg|j|jdddg|j|jd dddg|j|jdddg|j|jdd dgdS( Ns Class 1. Class 2. Class 1. Class 3 and 4. R)tclass_R4sClass 1.RNR5sClass 3 and 4.R6RD(RRR'(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyttest_find_all_by_classs """"cCs|jd}|jddtjd}|j|dg|jddtjd}|j|dg|jddtjd}|j|dgdS(Ns#Found ittgarRltosFound itR)so b(RR'R/R0R(RRKtf((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt0test_find_by_class_when_multiple_classes_present-scCs|jd}|j|jdtjddgd}|j|jd|gd}|j|jd|dgdS(NsFound itR)tbasFound itcSst|dkS(Ni(R&(tvalue((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pytbig_attribute_value@scSst|dkS(Ni(R&(Rs((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pytsmall_attribute_valueEs(RRR'R/R0(RRRtRu((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt:test_find_all_with_non_dictionary_for_attrs_finds_by_class;s(  cCs|jd}|jd\}}|j||g|jdd|j|g|jdd|j|g|jddd|j|g|jdd|jg|jdddS(Ns*R)R9R-Rlsfoo barsbar foo(RR'R(RRR)ta2((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt:test_find_all_with_string_for_attrs_finds_multiple_classesKs""cCsE|jd}tdidd6}|j|j|dgdS(Nsi Match. Non-match.RDR_RsMatch.(RR RR'(RRKtstrainer((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt'test_find_all_by_attribute_soupstrainerWs cCs5|jd}|j|jddddgdS(NsID present. No ID present. ID is empty.R)RsNo ID present.(RRR'tNone(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt$test_find_all_with_missing_attribute_s cCs5|jd}|j|jdtddgdS(NsID present. No ID present. ID is empty.Rs ID present.s ID is empty.(RRR'R1(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt$test_find_all_with_defined_attributegs cCsW|jd}ddg}|j|jdd||j|jdd|dS(Ns[Unquoted attribute. Quoted attribute.sUnquoted attribute.sQuoted attribute.RiR4(RRR'(RRKtexpected((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt$test_find_all_with_numeric_attributeps   cCs>|jd}|j|jddddgddgdS(Ns1 2 3 No ID.RR4R5R6(RRR'(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt(test_find_all_with_list_attribute_valuesys cCs>|jd}|j|jdtjdddgdS(NsOne a. Two as. Mixed as and bs. One b. No ID.Rs^a+$sOne a.sTwo as.(RRR'R/R0(RRK((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt5test_find_all_with_regular_expression_attribute_values cCs||jd}|j}|j|g|jddd|jg|jddd|jg|jddddS(NsfoobarfooR)R$R9R-(RR)RR'(RRR)((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt'test_find_by_name_and_containing_strings  "cCs;|jd}|j|jd|jddddS(Ns"foofooR)R$R9(RRR'(RR((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt=test_find_by_name_and_containing_string_when_string_is_buriedscCsc|jd}|j}|j|g|jdddd|jg|jdddddS(Ns"foofooRiR$R9iR-(RR)RR'(RRR)((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt,test_find_by_attribute_and_containing_strings %(RRR`RgRkRmRqRvRxRzR|R}RRRRRR(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyR^s          t TestIndexcBseZdZdZRS(sTest Tag.indexcCsk|jd}|j}x6t|jD]%\}}|j||j|q(W|jt|jddS(Nsh
Identical Not identical Identical Identical with child Also not identical Identical with child
i(RRXt enumeratetcontentsRtindext assertRaisest ValueError(RRKRXtitelement((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyt test_indexs   (RRR+R(((s7/usr/lib/python2.7/site-packages/bs4/tests/test_tree.pyRstTestParentOperationscBs_eZdZdZdZdZdZdZdZdZ dZ d Z RS( s;Test navigation and searching through an element's parents.cCs8tt|j|jd|_|jj|_dS(Ns1