�ɲɾ�����ӯ�����һ��ˣ��������С���˴��ͣ�������P���ҹ��ñ˽��ά�Բ��������˸߸ԣ�������ơ��ҹ��ñ�����ά�Բ���ˡ���˳^�ӣ������ӡ�
���ͯj�ӣ��ƺ���ӣ�
? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!? PNG ?%k25u25%fgd5n!PK 1]T002 2 Tagset.pmnu [ package HTML::Tagset;
use strict;
=head1 NAME
HTML::Tagset - data tables useful in parsing HTML
=head1 VERSION
Version 3.20
=cut
use vars qw( $VERSION );
$VERSION = '3.20';
=head1 SYNOPSIS
use HTML::Tagset;
# Then use any of the items in the HTML::Tagset package
# as need arises
=head1 DESCRIPTION
This module contains several data tables useful in various kinds of
HTML parsing operations.
Note that all tag names used are lowercase.
In the following documentation, a "hashset" is a hash being used as a
set -- the hash conveys that its keys are there, and the actual values
associated with the keys are not significant. (But what values are
there, are always true.)
=cut
use vars qw(
$VERSION
%emptyElement %optionalEndTag %linkElements %boolean_attr
%isHeadElement %isBodyElement %isPhraseMarkup
%is_Possible_Strict_P_Content
%isHeadOrBodyElement
%isList %isTableElement %isFormElement
%isKnown %canTighten
@p_closure_barriers
%isCDATA_Parent
);
=head1 VARIABLES
Note that none of these variables are exported.
=head2 hashset %HTML::Tagset::emptyElement
This hashset has as values the tag-names (GIs) of elements that cannot
have content. (For example, "base", "br", "hr".) So
C<$HTML::Tagset::emptyElement{'hr'}> exists and is true.
C<$HTML::Tagset::emptyElement{'dl'}> does not exist, and so is not true.
=cut
%emptyElement = map {; $_ => 1 } qw(base link meta isindex
img br hr wbr
input area param
embed bgsound spacer
basefont col frame
~comment ~literal
~declaration ~pi
);
# The "~"-initial names are for pseudo-elements used by HTML::Entities
# and TreeBuilder
=head2 hashset %HTML::Tagset::optionalEndTag
This hashset lists tag-names for elements that can have content, but whose
end-tags are generally, "safely", omissible. Example:
C<$HTML::Tagset::emptyElement{'li'}> exists and is true.
=cut
%optionalEndTag = map {; $_ => 1 } qw(p li dt dd); # option th tr td);
=head2 hash %HTML::Tagset::linkElements
Values in this hash are tagnames for elements that might contain
links, and the value for each is a reference to an array of the names
of attributes whose values can be links.
=cut
%linkElements =
(
'a' => ['href'],
'applet' => ['archive', 'codebase', 'code'],
'area' => ['href'],
'base' => ['href'],
'bgsound' => ['src'],
'blockquote' => ['cite'],
'body' => ['background'],
'del' => ['cite'],
'embed' => ['pluginspage', 'src'],
'form' => ['action'],
'frame' => ['src', 'longdesc'],
'iframe' => ['src', 'longdesc'],
'ilayer' => ['background'],
'img' => ['src', 'lowsrc', 'longdesc', 'usemap'],
'input' => ['src', 'usemap'],
'ins' => ['cite'],
'isindex' => ['action'],
'head' => ['profile'],
'layer' => ['background', 'src'],
'link' => ['href'],
'object' => ['classid', 'codebase', 'data', 'archive', 'usemap'],
'q' => ['cite'],
'script' => ['src', 'for'],
'table' => ['background'],
'td' => ['background'],
'th' => ['background'],
'tr' => ['background'],
'xmp' => ['href'],
);
=head2 hash %HTML::Tagset::boolean_attr
This hash (not hashset) lists what attributes of what elements can be
printed without showing the value (for example, the "noshade" attribute
of "hr" elements). For elements with only one such attribute, its value
is simply that attribute name. For elements with many such attributes,
the value is a reference to a hashset containing all such attributes.
=cut
%boolean_attr = (
# TODO: make these all hashes
'area' => 'nohref',
'dir' => 'compact',
'dl' => 'compact',
'hr' => 'noshade',
'img' => 'ismap',
'input' => { 'checked' => 1, 'readonly' => 1, 'disabled' => 1 },
'menu' => 'compact',
'ol' => 'compact',
'option' => 'selected',
'select' => 'multiple',
'td' => 'nowrap',
'th' => 'nowrap',
'ul' => 'compact',
);
#==========================================================================
# List of all elements from Extensible HTML version 1.0 Transitional DTD:
#
# a abbr acronym address applet area b base basefont bdo big
# blockquote body br button caption center cite code col colgroup
# dd del dfn dir div dl dt em fieldset font form h1 h2 h3 h4 h5 h6
# head hr html i iframe img input ins isindex kbd label legend li
# link map menu meta noframes noscript object ol optgroup option p
# param pre q s samp script select small span strike strong style
# sub sup table tbody td textarea tfoot th thead title tr tt u ul
# var
#
# Varia from Mozilla source internal table of tags:
# Implemented:
# xmp listing wbr nobr frame frameset noframes ilayer
# layer nolayer spacer embed multicol
# But these are unimplemented:
# sound?? keygen?? server??
# Also seen here and there:
# marquee?? app?? (both unimplemented)
#==========================================================================
=head2 hashset %HTML::Tagset::isPhraseMarkup
This hashset contains all phrasal-level elements.
=cut
%isPhraseMarkup = map {; $_ => 1 } qw(
span abbr acronym q sub sup
cite code em kbd samp strong var dfn strike
b i u s tt small big
a img br
wbr nobr blink
font basefont bdo
spacer embed noembed
); # had: center, hr, table
=head2 hashset %HTML::Tagset::is_Possible_Strict_P_Content
This hashset contains all phrasal-level elements that be content of a
P element, for a strict model of HTML.
=cut
%is_Possible_Strict_P_Content = (
%isPhraseMarkup,
%isFormElement,
map {; $_ => 1} qw( object script map )
# I've no idea why there's these latter exceptions.
# I'm just following the HTML4.01 DTD.
);
#from html4 strict:
#
#
#
#
#
#
#
#
#
#
=head2 hashset %HTML::Tagset::isHeadElement
This hashset contains all elements that elements that should be
present only in the 'head' element of an HTML document.
=cut
%isHeadElement = map {; $_ => 1 }
qw(title base link meta isindex script style object bgsound);
=head2 hashset %HTML::Tagset::isList
This hashset contains all elements that can contain "li" elements.
=cut
%isList = map {; $_ => 1 } qw(ul ol dir menu);
=head2 hashset %HTML::Tagset::isTableElement
This hashset contains all elements that are to be found only in/under
a "table" element.
=cut
%isTableElement = map {; $_ => 1 }
qw(tr td th thead tbody tfoot caption col colgroup);
=head2 hashset %HTML::Tagset::isFormElement
This hashset contains all elements that are to be found only in/under
a "form" element.
=cut
%isFormElement = map {; $_ => 1 }
qw(input select option optgroup textarea button label);
=head2 hashset %HTML::Tagset::isBodyMarkup
This hashset contains all elements that are to be found only in/under
the "body" element of an HTML document.
=cut
%isBodyElement = map {; $_ => 1 } qw(
h1 h2 h3 h4 h5 h6
p div pre plaintext address blockquote
xmp listing
center
multicol
iframe ilayer nolayer
bgsound
hr
ol ul dir menu li
dl dt dd
ins del
fieldset legend
map area
applet param object
isindex script noscript
table
center
form
),
keys %isFormElement,
keys %isPhraseMarkup, # And everything phrasal
keys %isTableElement,
;
=head2 hashset %HTML::Tagset::isHeadOrBodyElement
This hashset includes all elements that I notice can fall either in
the head or in the body.
=cut
%isHeadOrBodyElement = map {; $_ => 1 }
qw(script isindex style object map area param noscript bgsound);
# i.e., if we find 'script' in the 'body' or the 'head', don't freak out.
=head2 hashset %HTML::Tagset::isKnown
This hashset lists all known HTML elements.
=cut
%isKnown = (%isHeadElement, %isBodyElement,
map{; $_=>1 }
qw( head body html
frame frameset noframes
~comment ~pi ~directive ~literal
));
# that should be all known tags ever ever
=head2 hashset %HTML::Tagset::canTighten
This hashset lists elements that might have ignorable whitespace as
children or siblings.
=cut
%canTighten = %isKnown;
delete @canTighten{
keys(%isPhraseMarkup), 'input', 'select',
'xmp', 'listing', 'plaintext', 'pre',
};
# xmp, listing, plaintext, and pre are untightenable, and
# in a really special way.
@canTighten{'hr','br'} = (1,1);
# exceptional 'phrasal' things that ARE subject to tightening.
# The one case where I can think of my tightening rules failing is:
#
foo bar
baz quux ...
# ^-- that would get deleted.
# But that's pretty gruesome code anyhow. You gets what you pays for.
#==========================================================================
=head2 array @HTML::Tagset::p_closure_barriers
This array has a meaning that I have only seen a need for in
C, but I include it here on the off chance that someone
might find it of use:
When we see a "EpE" token, we go lookup up the lineage for a p
element we might have to minimize. At first sight, we might say that
if there's a p anywhere in the lineage of this new p, it should be
closed. But that's wrong. Consider this document:
foo
foo
foo
bar
The second p is quite legally inside a much higher p.
My formalization of the reason why this is legal, but this:
foo
bar
isn't, is that something about the table constitutes a "barrier" to
the application of the rule about what p must minimize.
So C<@HTML::Tagset::p_closure_barriers> is the list of all such
barrier-tags.
=cut
@p_closure_barriers = qw(
li blockquote
ul ol menu dir
dl dt dd
td th tr table caption
div
);
# In an ideal world (i.e., XHTML) we wouldn't have to bother with any of this
# monkey business of barriers to minimization!
=head2 hashset %isCDATA_Parent
This hashset includes all elements whose content is CDATA.
=cut
%isCDATA_Parent = map {; $_ => 1 }
qw(script style xmp listing plaintext);
# TODO: there's nothing else that takes CDATA children, right?
# As the HTML3 DTD (Raggett 1995-04-24) noted:
# The XMP, LISTING and PLAINTEXT tags are incompatible with SGML
# and derive from very early versions of HTML. They require non-
# standard parsers and will cause problems for processing
# documents with standard SGML tools.
=head1 CAVEATS
You may find it useful to alter the behavior of modules (like
C or C) that use C's
data tables by altering the data tables themselves. You are welcome
to try, but be careful; and be aware that different modules may or may
react differently to the data tables being changed.
Note that it may be inappropriate to use these tables for I
HTML -- for example, C<%isHeadOrBodyElement> lists the tagnames
for all elements that can appear either in the head or in the body,
such as "script". That doesn't mean that I am saying your code that
produces HTML should feel free to put script elements in either place!
If you are producing programs that spit out HTML, you should be
I familiar with the DTDs for HTML or XHTML (available at
C), and you should slavishly obey them, not
the data tables in this document.
=head1 SEE ALSO
L, L, L
=head1 COPYRIGHT & LICENSE
Copyright 1995-2000 Gisle Aas.
Copyright 2000-2005 Sean M. Burke.
Copyright 2005-2008 Andy Lester.
This program is free software; you can redistribute it and/or modify it
under the same terms as Perl itself.
=head1 ACKNOWLEDGEMENTS
Most of the code/data in this module was adapted from code written
by Gisle Aas for C, C, and
C. Then it was maintained by Sean M. Burke.
=head1 AUTHOR
Current maintainer: Andy Lester, C<< >>
=head1 BUGS
Please report any bugs or feature requests to
C, or through the web interface at
L. I will
be notified, and then you'll automatically be notified of progress on
your bug as I make changes.
=cut
1;
PK *1]He LinkExtor.pmnu [ package HTML::LinkExtor;
require HTML::Parser;
@ISA = qw(HTML::Parser);
$VERSION = "3.69";
=head1 NAME
HTML::LinkExtor - Extract links from an HTML document
=head1 SYNOPSIS
require HTML::LinkExtor;
$p = HTML::LinkExtor->new(\&cb, "http://www.perl.org/");
sub cb {
my($tag, %links) = @_;
print "$tag @{[%links]}\n";
}
$p->parse_file("index.html");
=head1 DESCRIPTION
I is an HTML parser that extracts links from an
HTML document. The I is a subclass of
I. This means that the document should be given to the
parser by calling the $p->parse() or $p->parse_file() methods.
=cut
use strict;
use HTML::Tagset ();
# legacy (some applications grabs this hash directly)
use vars qw(%LINK_ELEMENT);
*LINK_ELEMENT = \%HTML::Tagset::linkElements;
=over 4
=item $p = HTML::LinkExtor->new
=item $p = HTML::LinkExtor->new( $callback )
=item $p = HTML::LinkExtor->new( $callback, $base )
The constructor takes two optional arguments. The first is a reference
to a callback routine. It will be called as links are found. If a
callback is not provided, then links are just accumulated internally
and can be retrieved by calling the $p->links() method.
The $base argument is an optional base URL used to absolutize all URLs found.
You need to have the I module installed if you provide $base.
The callback is called with the lowercase tag name as first argument,
and then all link attributes as separate key/value pairs. All
non-link attributes are removed.
=cut
sub new
{
my($class, $cb, $base) = @_;
my $self = $class->SUPER::new(
start_h => ["_start_tag", "self,tagname,attr"],
report_tags => [keys %HTML::Tagset::linkElements],
);
$self->{extractlink_cb} = $cb;
if ($base) {
require URI;
$self->{extractlink_base} = URI->new($base);
}
$self;
}
sub _start_tag
{
my($self, $tag, $attr) = @_;
my $base = $self->{extractlink_base};
my $links = $HTML::Tagset::linkElements{$tag};
$links = [$links] unless ref $links;
my @links;
my $a;
for $a (@$links) {
next unless exists $attr->{$a};
(my $link = $attr->{$a}) =~ s/^\s+//; $link =~ s/\s+$//; # HTML5
push(@links, $a, $base ? URI->new($link, $base)->abs($base) : $link);
}
return unless @links;
$self->_found_link($tag, @links);
}
sub _found_link
{
my $self = shift;
my $cb = $self->{extractlink_cb};
if ($cb) {
&$cb(@_);
} else {
push(@{$self->{'links'}}, [@_]);
}
}
=item $p->links
Returns a list of all links found in the document. The returned
values will be anonymous arrays with the following elements:
[$tag, $attr => $url1, $attr2 => $url2,...]
The $p->links method will also truncate the internal link list. This
means that if the method is called twice without any parsing
between them the second call will return an empty list.
Also note that $p->links will always be empty if a callback routine
was provided when the I was created.
=cut
sub links
{
my $self = shift;
exists($self->{'links'}) ? @{delete $self->{'links'}} : ();
}
# We override the parse_file() method so that we can clear the links
# before we start a new file.
sub parse_file
{
my $self = shift;
delete $self->{'links'};
$self->SUPER::parse_file(@_);
}
=back
=head1 EXAMPLE
This is an example showing how you can extract links from a document
received using LWP:
use LWP::UserAgent;
use HTML::LinkExtor;
use URI::URL;
$url = "http://www.perl.org/"; # for instance
$ua = LWP::UserAgent->new;
# Set up a callback that collect image links
my @imgs = ();
sub callback {
my($tag, %attr) = @_;
return if $tag ne 'img'; # we only look closer at
push(@imgs, values %attr);
}
# Make the parser. Unfortunately, we don't know the base yet
# (it might be different from $url)
$p = HTML::LinkExtor->new(\&callback);
# Request document and parse it as it arrives
$res = $ua->request(HTTP::Request->new(GET => $url),
sub {$p->parse($_[0])});
# Expand all image URLs to absolute ones
my $base = $res->base;
@imgs = map { $_ = url($_, $base)->abs; } @imgs;
# Print them out
print join("\n", @imgs), "\n";
=head1 SEE ALSO
L, L, L, L
=head1 COPYRIGHT
Copyright 1996-2001 Gisle Aas.
This library is free software; you can redistribute it and/or
modify it under the same terms as Perl itself.
=cut
1;
PK *1],91 1
PullParser.pmnu [ package HTML::PullParser;
require HTML::Parser;
@ISA=qw(HTML::Parser);
$VERSION = "3.57";
use strict;
use Carp ();
sub new
{
my($class, %cnf) = @_;
# Construct argspecs for the various events
my %argspec;
for (qw(start end text declaration comment process default)) {
my $tmp = delete $cnf{$_};
next unless defined $tmp;
$argspec{$_} = $tmp;
}
Carp::croak("Info not collected for any events")
unless %argspec;
my $file = delete $cnf{file};
my $doc = delete $cnf{doc};
Carp::croak("Can't parse from both 'doc' and 'file' at the same time")
if defined($file) && defined($doc);
Carp::croak("No 'doc' or 'file' given to parse from")
unless defined($file) || defined($doc);
# Create object
$cnf{api_version} = 3;
my $self = $class->SUPER::new(%cnf);
my $accum = $self->{pullparser_accum} = [];
while (my($event, $argspec) = each %argspec) {
$self->SUPER::handler($event => $accum, $argspec);
}
if (defined $doc) {
$self->{pullparser_str_ref} = ref($doc) ? $doc : \$doc;
$self->{pullparser_str_pos} = 0;
}
else {
if (!ref($file) && ref(\$file) ne "GLOB") {
require IO::File;
$file = IO::File->new($file, "r") || return;
}
$self->{pullparser_file} = $file;
}
$self;
}
sub handler
{
Carp::croak("Can't set handlers for HTML::PullParser");
}
sub get_token
{
my $self = shift;
while (!@{$self->{pullparser_accum}} && !$self->{pullparser_eof}) {
if (my $f = $self->{pullparser_file}) {
# must try to parse more from the file
my $buf;
if (read($f, $buf, 512)) {
$self->parse($buf);
} else {
$self->eof;
$self->{pullparser_eof}++;
delete $self->{pullparser_file};
}
}
elsif (my $sref = $self->{pullparser_str_ref}) {
# must try to parse more from the scalar
my $pos = $self->{pullparser_str_pos};
my $chunk = substr($$sref, $pos, 512);
$self->parse($chunk);
$pos += length($chunk);
if ($pos < length($$sref)) {
$self->{pullparser_str_pos} = $pos;
}
else {
$self->eof;
$self->{pullparser_eof}++;
delete $self->{pullparser_str_ref};
delete $self->{pullparser_str_pos};
}
}
else {
die;
}
}
shift @{$self->{pullparser_accum}};
}
sub unget_token
{
my $self = shift;
unshift @{$self->{pullparser_accum}}, @_;
$self;
}
1;
__END__
=head1 NAME
HTML::PullParser - Alternative HTML::Parser interface
=head1 SYNOPSIS
use HTML::PullParser;
$p = HTML::PullParser->new(file => "index.html",
start => 'event, tagname, @attr',
end => 'event, tagname',
ignore_elements => [qw(script style)],
) || die "Can't open: $!";
while (my $token = $p->get_token) {
#...do something with $token
}
=head1 DESCRIPTION
The HTML::PullParser is an alternative interface to the HTML::Parser class.
It basically turns the HTML::Parser inside out. You associate a file
(or any IO::Handle object or string) with the parser at construction time and
then repeatedly call $parser->get_token to obtain the tags and text
found in the parsed document.
The following methods are provided:
=over 4
=item $p = HTML::PullParser->new( file => $file, %options )
=item $p = HTML::PullParser->new( doc => \$doc, %options )
A C can be made to parse from either a file or a
literal document based on whether the C or C option is
passed to the parser's constructor.
The C passed in can either be a file name or a file handle
object. If a file name is passed, and it can't be opened for reading,
then the constructor will return an undefined value and $! will tell
you why it failed. Otherwise the argument is taken to be some object
that the C can read() from when it needs more data.
The stream will be read() until EOF, but not closed.
A C can be passed plain or as a reference
to a scalar. If a reference is passed then the value of this scalar
should not be changed before all tokens have been extracted.
Next the information to be returned for the different token types must
be set up. This is done by simply associating an argspec (as defined
in L) with the events you have an interest in. For
instance, if you want C tokens to be reported as the string
C<'S'> followed by the tagname and the attributes you might pass an
C-option like this:
$p = HTML::PullParser->new(
doc => $document_to_parse,
start => '"S", tagname, @attr',
end => '"E", tagname',
);
At last other C options, like C, and
C, can be passed in. Note that you should not use the
I_h options to set up parser handlers. That would confuse the
inner logic of C.
=item $token = $p->get_token
This method will return the next I found in the HTML document,
or C at the end of the document. The token is returned as an
array reference. The content of this array match the argspec set up
during C construction.
=item $p->unget_token( @tokens )
If you find out you have read too many tokens you can push them back,
so that they are returned again the next time $p->get_token is called.
=back
=head1 EXAMPLES
The 'eg/hform' script shows how we might parse the form section of
HTML::Documents using HTML::PullParser.
=head1 SEE ALSO
L, L
=head1 COPYRIGHT
Copyright 1998-2001 Gisle Aas.
This library is free software; you can redistribute it and/or
modify it under the same terms as Perl itself.
=cut
PK *1]Ӳ: : Entities.pmnu [ package HTML::Entities;
=encoding utf8
=head1 NAME
HTML::Entities - Encode or decode strings with HTML entities
=head1 SYNOPSIS
use HTML::Entities;
$a = "Våre norske tegn bør æres";
decode_entities($a);
encode_entities($a, "\200-\377");
For example, this:
$input = "vis-à-vis Beyoncé's naïve\npapier-mâché résumé";
print encode_entities($input), "\n"
Prints this out:
vis-à-vis Beyoncé's naïve
papier-mâché résumé
=head1 DESCRIPTION
This module deals with encoding and decoding of strings with HTML
character entities. The module provides the following functions:
=over 4
=item decode_entities( $string, ... )
This routine replaces HTML entities found in the $string with the
corresponding Unicode character. Unrecognized entities are left alone.
If multiple strings are provided as argument they are each decoded
separately and the same number of strings are returned.
If called in void context the arguments are decoded in-place.
This routine is exported by default.
=item _decode_entities( $string, \%entity2char )
=item _decode_entities( $string, \%entity2char, $expand_prefix )
This will in-place replace HTML entities in $string. The %entity2char
hash must be provided. Named entities not found in the %entity2char
hash are left alone. Numeric entities are expanded unless their value
overflow.
The keys in %entity2char are the entity names to be expanded and their
values are what they should expand into. The values do not have to be
single character strings. If a key has ";" as suffix,
then occurrences in $string are only expanded if properly terminated
with ";". Entities without ";" will be expanded regardless of how
they are terminated for compatibility with how common browsers treat
entities in the Latin-1 range.
If $expand_prefix is TRUE then entities without trailing ";" in
%entity2char will even be expanded as a prefix of a longer
unrecognized name. The longest matching name in %entity2char will be
used. This is mainly present for compatibility with an MSIE
misfeature.
$string = "foo bar";
_decode_entities($string, { nb => "@", nbsp => "\xA0" }, 1);
print $string; # will print "foo bar"
This routine is exported by default.
=item encode_entities( $string )
=item encode_entities( $string, $unsafe_chars )
This routine replaces unsafe characters in $string with their entity
representation. A second argument can be given to specify which characters to
consider unsafe. The unsafe characters is specified using the regular
expression character class syntax (what you find within brackets in regular
expressions).
The default set of characters to encode are control chars, high-bit chars, and
the C<< < >>, C<< & >>, C<< > >>, C<< ' >> and C<< " >> characters. But this,
for example, would encode I the C<< < >>, C<< & >>, C<< > >>, and C<< "
>> characters:
$encoded = encode_entities($input, '<>&"');
and this would only encode non-plain ascii:
$encoded = encode_entities($input, '^\n\x20-\x25\x27-\x7e');
This routine is exported by default.
=item encode_entities_numeric( $string )
=item encode_entities_numeric( $string, $unsafe_chars )
This routine works just like encode_entities, except that the replacement
entities are always C<&#xI;> and never C<&I;>. For
example, C returns "rôle", but
C returns "rôle".
This routine is I exported by default. But you can always
export it with C