This section describes the status of this document at the
time of its publication. Other documents may supersede this
document. The latest status of this document series is maintained
at the W3C.
This document has been developed by W3C's Technical Architecture Group
(TAG) (charter).
At their 23 June 2003
teleconference, the TAG agreed that the editor should request
publication of this draft with the approval of Dan Connolly and Tim
Bray. A complete list of changes since the
previous Working Draft is available on the Web.
This draft remains incomplete; sections 1, 2, and 3 are the most
developed; 4 the least. The TAG has published a number of findings that
address specific architecture issues. Parts of those findings may
appear in subsequent drafts. Please also consult the list of
issues under consideration by the TAG.
This draft includes some editorial notes and also references to
open TAG
issues. These do not represent all open issues in the document.
They are expected to disappear from future drafts.
Publication as a Working Draft does not imply endorsement by the
W3C Membership. This is a draft document and may be updated,
replaced or obsoleted by other documents at any time. It is
inappropriate to cite this document as other than "work in
progress."
The latest information regarding patent disclosures
related to this document is available on the Web. As of this
publication, there are no disclosures.
Please send comments on this document to the public W3C TAG
mailing list www-tag@w3.org (archive).
A list of current
W3C Recommendations and other technical documents can be found
at the W3C Web site.
Web architecture starts with Uniform Resource Identifiers (URI),
defined by "Uniform Resource Identifiers (URI): Generic Syntax" [URI]. Parties who wish to communicate
about something will establish a shared vocabulary, i.e. a shared
set of bindings between identifiers and things. This shared
vocabulary has a tangible value: it reduces the cost of
communication. The ability to use common identifiers across
communities is what motivates global naming in Web
Architecture.
URIs identify resources. When a representation of one resource
refers to another resource with a URI, a link is formed between the two
resources. The networked information system is built of linked
resources, and the large-scale effect is a shared information
space. The value of the Web grows exponentially as a function of
the number of linked resources (the "network effect").
Principle
Use URIs: All important resources SHOULD be
identified by a URI.3
There are many benefits to making resources identifiable by URI.
Some are by design (e.g., linking, bookmarking, and caching),
others (e.g., global search services) were not predicted.4
An important aspect of communication is to be able to establish
when two parties are talking about the same thing. In the context
of the Web, this means when two parties identify the same resource.
The most common way to establish that two parties are identifying
the same resource is to compare the spelling (i.e., as strings) of
the identifiers the parties are using. Section 6 of [URI] discusses this type of analysis.
In that specification, determination of equivalence or difference
of URIs is based on string comparison, perhaps augmented by
reference to additional rules provided by URI scheme definitions
(e.g., for HTTP URIs, the authority component is case-insensitive).
Depending on the application, an agent may invest more processing
effort to reduce the likelihood of a false negative (i.e., two URIs
identify the same resource, but that was not detected).
There may be other ways to establish that two parties are
identifying the same resource that are not based on string
comparison; see the section on future directions for determining that two URIs
identify the same resource.
Editor's note: Dan Connolly has
suggested the term "coreference" instead of "equivalence" to
communicate that two URIs are referring to the same resource.
Agents that reach conclusions about identity beyond what they
are licensed to do (e.g., by specification, or community
convention, or site-specific convention) take responsibility for
any problems that result. For instance, agents should not assume
that http://weather.example.com/Oaxaca and
http://weather.example.com/oaxaca identify the same
resource, since none of the specifications involved states that the
path part of an HTTP URI is case-insensitive. Web servers may vary
in how they are configured to handle case-sensitivity. Agents that
assume these URIs identify the same resource take responsibility
for any resulting problems.
Although it is possible to determine that two URIs are
equivalent, it is generally not possible by mere inspection of two
URIs to be sure that they identify different resources. Web
architecture does not constrain resources to be uniquely named.
Good practice
Spelling URIs: If an agent has been provided
with a URI to refer to a resource, the agent SHOULD use the
spelling of the URI as it was originally provided.
To help parties know when they are referring to the same
resource, it follows that URI producers should be conservative
about the number of different URIs they produce for the same
resource. For instance, the parties responsible for
weather.example.com have no reason to use both
http://weather.example.com/Oaxaca and
http://weather.example.com/oaxaca to refer to the same
resource; agents will not detect the equivalence relationship. In
this case, one URI should be chosen and used consistently. See
section 6.3 of [URI] for further
advice on how to reduce the risk of false negatives.
URI consumers cannot, in general, determine the meaning of a
resource by inspection of a URI that identifies it. In our travel scenario, the example URI
(http://weather.example.com/oaxaca) suggests that the
identified resource has something to do with the weather in Oaxaca.
Although short, meaningful URIs benefit people, URI consumers must
not rely on the URI string to communicate the meaning of a
resource. A site reporting the weather in Oaxaca could just as
easily be identified by the URI
http://vjc.example.com/315. And the URI
http://weather.example.com/vancouver might identify
the resource "my photo album." See the section on retrieving a representation for
information about how the meaning of a resource is conveyed.
Editor's note: When finding
available on URI opacity, link from here.
In the URI http://weather.example.com/, the "http"
that appears before the colon (":") is a URI scheme name. There are
other scheme names, such as "mailto" and "ftp". It is common to
classify URIs by scheme; a URI with scheme "http" is called an
"HTTP URI."
Each URI begins with a URI scheme name. The scheme name
corresponds to a specification for assigning identifiers within
that scheme. As such, the URI syntax is a federated and extensible
naming system wherein each scheme's specification may further
restrict the syntax and semantics of identifiers using that scheme.
Furthermore, the URI scheme specification specifies how an agent
can dereference the
URI.
Several URI schemes incorporate identification mechanisms that
pre-date the Web into this syntax:
- MAILTO URIs identify mailboxes:
mailto:nobody@example.org
- FTP URIs identify identify ftp files and directories:
ftp://example.org/aDirectory/aFile
- NEWS URIs identify USENET newsgroups:
news:comp.infosystems.www
- TEL URIs identify terminals on the telephone network:
tel:+1-816-555-1212
Other URI schemes have been introduced since the advent of the
Web, including those introduced as a consequence of new protocols.
Examples of URIs for these schemes include:
- HTTP URIs:
http://www.example.org/something?with=arg1;and=arg2
- LDAP URIs:
ldap://ldap.itd.umich.edu/c=GB?objectClass?one
- URNs:
urn:oasis:SAML:1.0
The Internet Assigned Numbers Authority
(IANA) maintains a registry [IANASchemes] of mapping between URI scheme
names and their specifications. For instance, the IANA registry
indicates that the "http" scheme is defined by [RFC2616]. The process for registration of new
URI schemes is defined by RFC2717.
Since many aspects of URI processing are scheme-dependent, and
since a huge amount of deployed software already processes URIs of
well-known schemes, the cost of introduction of new URI schemes is
high. We note in passing that even more expensive than introducing
a new URI scheme is introducing a new identification mechanism for
the Web; this is considered prohibitively expensive.
Good practice
New URI schemes: Authors of
specifications SHOULD avoid introducing new URI schemes when
existing schemes can be used to meet the goals of the
specifications.
Consider our travel
scenario: should the authority providing information about the
weather in Oaxaca register a new URI scheme "weather" for the
identification of resources related to the weather? They might then
publish URIs such as
weather://travel.example.com/oaxaca. While the Web
Architecture allows the definition of new schemes, there is a cost
to registration and especially deployment of new schemes. When an
agent dereferences such a URI, if what really happens is that HTTP
GET is invoked to retrieve an HTML representation of the resource,
then an HTTP URI would have sufficed. If a URI scheme exists that
meets the needs of an application, designers should use it rather
than invent one. Furthermore, designers should expect that it will
prove useful to be able to share a URI across applications, even if
that utility is not initially evident.
If the motivation behind registering a new scheme is to allow an
agent to launch a particular application when retrieving a
representation, such dispatching can be accomplished at lower
expense by registering a new MIME type instead. Reasons for this
include:
- The more resource metadata is included in a URI, the more
fragile the URI becomes (e.g., sensitive to changes in
representation). Designers should choose URI schemes that allow
them to keep their options open.
- At this time, the registration process for new URI schemes is
more strict than for the registration of new media types.
Editor's note: When finding
available based on Tim Bray's discussion of this topic, link from
here.
The use of unregistered URI schemes is discouraged for a number
of reasons:
- There is no generally accepted way to locate the scheme
specification.
- Someone else may be using the scheme for other purposes.
- One should not expect that general purpose software will do
anything useful with URIs of this scheme; the network effect is
lost.
In the URI http://weather.example.com/, the string
weather.example.com (between "//" and the next "/")
called the authority component. Many URI schemes include a
hierarchical element for a naming authority such that governance of
the name space defined by the remainder of the URI is delegated to
that authority (which may, in turn, delegate it further). The
generic syntax provides a common means for distinguishing an
authority based on a registered domain name or server address. See
section 3.2 of [URI] for more
information about the authority portion of a URI.
How authority is delegated depends on the URI scheme. The
deployment and use of different URI schemes may require varying
degrees of central coordination and administration. For example,
MAILTO, FTP, and HTTP URIs depend on the use of the DNS and IANA
infrastructure; see "ICP-1: Internet Domain Name System Structure
and Delegation" [IANAICP1] for
more information about how the IANA manages delegation of domain
names.
Successful communication between two parties about a piece of
information relies on shared understanding of the meaning of the
information. On the Web, thousands of independent parties can
identify and communicate about a Web resource. To give these
parties the confidence that they are all talking about the same
thing when they refer to "the resource identified by the following
URI ..." the design choice for the Web is, in general, that the
owner of a resource assigns its authoritative meaning and the URIs
that refer to it. See the draft TAG finding
"Client
handling of MIME headers" for related discussion.
In our travel scenario, the
agent responsible for weather.example.com has license
to assign the meaning of the resource and to create the
authoritative representations of this resource.
In our travel scenario the
server returns an XHTML representation when Dan dereferences the
URI http://weather.example.com/oaxaca. Then, by
navigating within the XHTML content, Dan finds that the URI
http://weather.example.com/oaxaca#tom refers to
information about tomorrow's weather in Oaxaca. This URI includes
the fragment identifier "tom" (the string after the "#").
The fragment
identifier component of a URI allows indirect
identification of a secondary resource, by
reference to a primary resource and additional identifying
information that is selective with respect to that resource. The
identified secondary resource may be some portion or subset of the
primary resource, some view on representations of the primary
resource, or some other resource that is merely named with respect
to the primary resource.
Although the generic URI syntax allows any URI to end with a
fragment identifier, some URI schemes do not specify the use of
fragment identifiers. For instance, fragment identifier semantics
are not specified for MAILTO URIs.
For URI schemes that do specify the use of fragment identifiers,
the syntax and semantics of those identifiers is defined by the set
of representations that
might result from a retrieval action on the
primary resource. The presence of a fragment identifier component
in a URI does not imply that a retrieval action will take
place.
Interpretation of the fragment identifier during a retrieval
action is performed solely by the user agent; the fragment
identifier is not passed to other systems during the process of
retrieval. This means that some intermediaries in the Web
architecture (e.g., proxies) have no effect on fragment identifiers
and that redirection (in HTTP [RFC2616], for example) does not account for
them.
Suppose that the managers of weather.example.com
provide a visual map of the meteorological conditions in Oaxaca as
part of the representation served for
http://weather.example.com/oaxaca. They might encode
the same visual map in a number of image formats to meet different
needs (e.g., they might serve PNG, SVG, and JPEG/JFIF). Dan's user
agent and the server engage in HTTP content negotiation, so that
Dan receives the best image format his user agent can handle. The
URI http://weather.example.com/oaxaca/map#zicatela
refers to a portion of the weather map that shows the Zicatela
Beach, where Dan intends to go surfing. This URI makes sense for
the SVG representation, since SVG defines fragment identifier
semantics. However, the URI does not make sense for the PNG and
JPEG/JFIF representations; those specifications do not define
fragment identifier semantics.
Good practice
Content negotiation with fragments:
Authors SHOULD NOT use HTTP content negotiation for different media
types that have incompatible fragment identifier semantics.
Given a URI, a system may attempt to perform a variety of
operations on the resource, as might be characterized by such words
as "access", "update", "replace", or "find attributes". Such
operations are defined by the formats and protocols that make use
of URIs. The URI specification (in [URI], section 1.2.2) defines the following terms
related to interactions through a URI.
- Resolution
- The process of determining an access mechanism and the
appropriate parameters necessary to dereference a URI; such
resolution may require several iterations.
- Dereference
- To dereference a URI is to use an access mechanism to perform
an action on the URI's resource.
- Retrieval
- A URI dereference that causes an agent to retrieve a
representation of the associated resource.
During URI resolution, an agent applies in succession a finite
set of relevant specifications, beginning with the specification of
the context in which the URI is found (e.g., a format or protocol
specification, or an application). Any one of these specifications
may define more than one access mechanism (e.g., the HTTP protocol
defines a number of access methods, including GET, HEAD, and POST).
Note that the information governing the choice of access mechanism
may be found in the context, not the URI itself (e.g., the choice
of HTTP GET v. HTTP HEAD). The draft TAG finding
"URIs,
Addressability, and the use of HTTP GET and POST."
discusses issues surrounding multiple access mechanisms and the
relation to URI addressability.
Some URI schemes (e.g., the URN scheme [RFC 2141]) do not define dereference
mechanisms.
TAG issue metadataInURI-31:
Should metadata (e.g., versioning information) be encoded in
URIs?
TAG issue siteDate-26:
Web site metadata improving on robots.txt, w3c/p3p and favicon
etc.
One of the most important actions on the Web is to retrieve a representation of a
resource (for example, by using HTTP GET). As stated above, the authority responsible for a
URI determines what the URI identifies and which
representations are used for interaction with the resource. The
representations communicate the meaning of the resource.
Good practice
Resource descriptions: Owners of
important resources SHOULD make available representations that
communicate the meaning of those resources.
As an example of representation retrieval, suppose that the URI
http://weather.example.com/budapest is used within an
a element of an SVG document. The sequence of
specifications applied is:
- The SVG 1.0 Recommendation [SVG10], which imports the link semantics defined
by XLink 1.0 [XLink10]. Section
17.1 of the SVG specification suggests that interaction with an
a link involves retrieving a representation of a
resource, identified by the XLink href attribute: "By
activating these links (by clicking with the mouse, through
keyboard input, and voice commands), users may visit these
resources."
- The attribute
xlink:href is defined in section 5.4
of the XLink 1.0 [XLink10]
specification states that "The value of the href attribute must be
a URI reference as defined in [IETF RFC 2396], or must result in a
URI reference after the escaping procedure described below is
applied."
- The URI specification [URI]
states that "Each URI begins with a scheme name that refers to a
specification for assigning identifiers within that scheme." The
URI scheme name in this example is "http".
- To find out what specification defines the "http" scheme, we
look up the mapping in [IANASchemes]; the "http" URI scheme is
defined in the HTTP/1.1 specification (RFC 2616 [RFC2616], section 3.2.2).
- The HTTP/1.1 specification defines several access mechanisms.
In this SVG context, the user agent employs the GET method (defined
in section 9.3 of [RFC2616]) to
retrieve the representation. The HTTP/1.1 specification explains
how the server constructs the response (section 6 of [RFC2616]), including the media
type of the representation. Section 1.4 states "HTTP communication
usually takes place over TCP/IP connections." Though not shown in
this example, the user agent would continue this process by
implementing those specifications.
- The user agent interprets the returned representation according
to the media type, but looking up which specification defines the
media type in the [IANAMIME]
registry.
Note that, in general, one cannot determine the media type(s) of
representation(s) of a resource by inspecting a URI for that
resource. For example, do not assume that all representations of
http://example.com/page.html are HTML. The HTTP
protocol does not constrain the media type based on the path
component of the URI; the server is free to return a PNG image
representation.
Dan's retrieval of weather information qualifies as a "safe"
interaction; a safe interaction is one
where the user agent does not commit to anything beyond the
interaction and is not responsible for any consequences other than
the interaction itself (e.g., a read-only query or lookup). Other
Web interactions resemble orders more than queries. These unsafe interactions may
cause a change to the state of a resource; the user may be held
responsible for the consequences of these interactions. Unsafe
interactions include subscription services, posting to a list, or
modifying a database.
Safe interactions are important because these are interactions
where users can browse with confidence and where software programs
(e.g., search engines and browsers that pre-cache data for the
user) can follow links safely. Users (or software agents acting on
their behalf) do not commit themselves to anything by querying a
resource or following a link.
Principle
Safe retrieval: Agents do not incur
obligations by retrieving a representation.
For instance, suppose in our travel scenario that the managers of
weather.example.com offer a monthly newsletter
available by subscription. It is incorrect and
harmful to publish a page
http://example.com/oxaca/aboutNewsLetter that states
"... terms and conditions..." with a link to
http://example.com/oxaca/newsLetter because search
services may link directly to
http://example.com/oxaca/newsLetter and readers that
follow such links may not have seen, let alone agreed to, the terms
and conditions.
For more information about safe and unsafe operations using HTTP
GET and POST, and handling security concerns around the use of HTTP
GET, see the draft TAG finding "URIs,
Addressability, and the use of HTTP GET and POST."
The value of a URI increases with the predictability of
interactions using that URI.
Good practice
URI Persistence: Parties responsible for a
URI SHOULD service that URI predictably and consistently.
Service breakdowns include:
- No service available (i.e., dereferencing fails).
- Inconsistent representations served. Note the difference
between a resource owner changing representations predictably in
light of the nature of the resource (e.g., the weather in Oaxaca
changes) and the owner changing representations arbitrarily.
- Improper use of content negotiation, such as serving two images
as equivalent through HTTP content negotiation, where one image
represents a square and the other a circle.
There are strong social expectations that once a URI identifies
a particular resource, it should continue indefinitely to refer to
that resource; this is called URI persistence. URI
persistence is always a matter of policy and commitment on the part
of authorities servicing URIs rather than a constraint imposed by
technological means.
URI persistence also improves when ambiguity is removed about
what the URI identifies. For instance, saying that the URI
http://www.example.com/moby identifies "Moby Dick" can
lead to confusion because this might be interpreted as any one of
the following very distinct resources: a particular printing of
this work (say, by ISBN), or the work itself in an abstract sense
(for example, using RDF), or the fictional white whale, or a
particular copy of the book on the shelves of a library (via the
Web interface of the library's online catalog), or the record in
the library's electronic catalog which contains the metadata about
the work, or the Gutenberg project's online
version. Similarly, one should not use the same URI to refer to
a person and to that person's mailbox.
Ambiguous descriptions of what a URI identifies increase the
likelihood that two parties will think the same URI identifies
different resources, and thus that the parties will use the URI
inconsistently. This can be costly, as in the case of two databases
in which the same URI is used inconsistently; merging the two
databases might lead to confusion or errors.
HTTP [RFC2616] has been
designed to help service URIs. For example, HTTP redirection (via
some of the 3xx response codes) permits servers to tell an agent
that further action needs to be taken by the agent in order to
fulfill the request (e.g., the resource has been assigned a new
URI). In addition, content negotiation also promotes consistency,
as a site manager would not be required to define new URIs for each
new format that is supported, as would be the case with protocols
that don't support content negotiation, such as FTP.
For more discussion about URI persistence, refer to [Cool].5
As we have seen, identification of a resource on the Web is
distinct from interacting with that resource. It is reasonable to
control access to the resource (e.g., for security reasons), but it
is unreasonable to prohibit others from merely identifying the
resource.
As an analogy: A building might have a policy that the public
may only enter via the main front door, and only during business
hours. People employed in the building and in making deliveries to
it might use other doors as appropriate. Such a policy would be
enforced by a combination of security personnel and mechanical
devices such as locks and pass-cards. One would not enforce this
policy by hiding some of the building entrances, nor by requesting
legislation requiring the use of the front door and forbidding
anyone to reveal the fact that there are other doors to the
building.
In the travel scenario,
imagine that Dan and Norm both subscribe to the
weather.example.com newsletter. Dan wishes to point
out an article of particular interest to Norm, using a URI. The
managers of weather.example.com can offer Dan and Norm
the benefits of URIs (e.g., bookmarking and linking) and still
control access to the newsletter by authorized parties. The Web
provides several mechanisms to control access to resources, none of
which relies on hiding or suppressing URIs for those resources. For
more information on identification and access control, please refer
to the TAG finding "'Deep
Linking' in the World Wide Web."
The integration of internationalized identifiers (i.e., composed
of characters beyond those allowed by [URI]) into the Web Architecture is an important and
open issue. See TAG issue IRIEverywhere-27
for discussion about work going on in this area.
Emerging Semantic Web technologies, including "DAML+OIL" [DAMLOIL] and "Web Ontology
Language (OWL)" [OWL10], define
RDF properties such as equivalentTo and
FunctionalProperty to state -- or at least claim --
formally that two URIs identify the same resource.
There has been some discussion but no agreement that new access
protocols should provide a means to convert fragment identifiers
according to media type.
Fragment identifier semantics may differ among formats. See
related TAG issues httpRange-14
and RDFinXHTML-35
and
abstractComponentRefs-37.
The Dynamic Delegation Discovery System
(DDDS) ([RFC3401] and related RFCs) is used to implement
lazy binding of strings to data, in order to support dynamically
configured delegation systems. This system is designed to allow
resolution of any type of URI, in particular URNs.
One area of work involves the creation of globally unique
identifiers in a file-sharing system without centralized or
hierarchical administration.
A representation is data that
represents or describes the state of a resource. It consists
of:
- Electronic data expressed in one or more formats (e.g., XHTML,
CSS, PNG, XLink, RDF/XML, and SMIL animation) used separately or in
combination.
- Metadata about the representation, such as the Internet Media
Type (defined in RFC 2046 [RFC2046]). The Internet Media Type is the key
to the correct interpretation of a resource representation, and
governs the handling of fragment
identifiers. When transferred by a Web protocol, a representation often includes
metadata about both the representation and the message bearing the
representation (for example, some HTTP headers).
Web agents use representations to modify as well as read
resource state.
In our previous travel
scenario the representation Dan receives (and whether he
receives one at all) depends on a number of factors, including:
- Whether the agents responsible for
weather.example.com respond to requests at all;
- Whether the agents responsible for
weather.example.com make available one or more
representations for the resource identified by
http://weather.example.com/oaxaca;
- Whether Dan has access privileges to such a
representation;
- If the agents responsible for
weather.example.com
have provided more than one representation (in different formats
such as HTML, PNG, or RDF, in different languages such as English
and Spanish, etc.), the result representation may depend on
negotiation between the user agent and server that occurs as part
of the HTTP transaction.
- When Dan made the request. Since the weather in Oaxaca changes,
Dan should expect that representations will change over time.
We discuss these issues in more detail below.
As discussed above, the owner of a resource assigns its
authoritative meaning and the URIs that refer to it. This meaning
is communicated in part through metadata that is part of the
representation, notably the Internet Media Type. At times there may
be inconsistencies between metadata and what is specified in a
format. Examples of inconsistencies between headers and format data
that have been observed on the Web include:
- The Unicode encoding of a message body (XML document) is
inconsistent with the value of the charset parameter in the message
headers.
- The namespace of the root element of a message body (XML
document) is inconsistent with the value of the content type header
in the message headers.
User agents should detect such inconsistencies but should not
resolve them without involving the user (e.g., by securing
permission or at least providing notification). User agents must
not silently ignore authoritative server metadata.
See the draft TAG finding "Client
handling of MIME headers" for more in-depth discussion
and examples.
The Web can be used to interchange resource representations in
any format. This flexibility is important, since there is
continuing progress in the development of new data formats for new
applications and the refinement of existing ones.
For a format to be usefully interoperable between two parties,
the parties must have a shared understanding of its syntax and
semantics. This is not to imply that a sender of data can
count on constraining its treatment by a receiver; simply that
making good use of electronic data usually requires knowledge of
its designers' intentions.
For a format to be widely interoperable across the Web:
- There SHOULD be a normative specification which is a stable and
widely available Web resource.
- The format specification SHOULD have an officially registered
Internet media type (see the [IANAMIME] registry). See TAG finding finding
"Internet Media
Type registration, consistency of use."
- The format specification SHOULD define the syntax and semantics
of fragment identifiers.
- The format specification SHOULD allow Web-wide linking, not
just internal document linking.
- The format specification SHOULD allow authors to use URIs
without constraining them to a limited set of URI schemes.
- The format specification MAY use Qualified Names (QNames) for
identifiers in content, but format designers should be aware of the
limitations and interoperability risks associated with QNames. See
the TAG finding "Using QNames as
Identifiers in Content" for more information.
Although the Web architecture allows for the deployment of new
data formats, the creation and deployment of new formats (and
software able to handle them) can be very expensive. Thus, before
inventing a new data format, designers should carefully consider
re-using one that is already available. For example, if a format is
required to contain human-readable text with embedded hyperlinks,
it is almost certainly better to use HTML for this purpose than to
invent a new format.
As noted above, the utility of data formats starts with the
availability of a normative specification. Some of the desirable
characteristics of a format include:
- Attention to Programmer Needs
- A specification that is written to meet the needs of
programmers is more likely to be implemented, and thus more likely
to be useful. In particular, the specification SHOULD be in part
formal and mathematical, rather than relying exclusively on
narrative.
- Attention to Author Needs
- Formats that allow authors to intuitively and efficiently
address problems they are trying to solve, that can be readily
learned, that are simple to use, and that are interoperable are
more likely to be adopted by authors and authoring tool
developers.
- Attention to User Needs
- A number of characteristics of a format specification will
address the needs of end users, and doing so will create a market
for the format. Attention to issues of accessibility,
internationalization, and device-independence will tend to make a
format specification more flexible.
- Attention to Error-Handling
- Given that representations are generated by humans (either
coded directly or mediated by an authoring tool) and then
transmitted in a heterogeneous network, it is inevitable that
errors will occur. Specifications of data formats SHOULD be clear
about behavior in the presence of errors. It is reasonable to
specify that errors should be worked around, or should result in
the termination of a transaction or session. It is not acceptable
for the behavior in the face of errors to be left unspecified. See
the draft TAG finding "Client
handling of MIME headers" for more discussion about
error reporting.
- Issue errorHandling-20
- Information hiding
- When designing specifications that address independent
functions of a system, references between the specifications are in
general harmful. They are harmful because they impede the
independent evolution of the specifications.
- For example, it is a strength of XML that XPath cannot query
the HTTP header. It is a strength of HTTP that it does not refer to
details of the underlying TCP to the extent that it cannot be run
over a different transport service. Similarly, the RDF data graph
has a significance that is independent of the actual
serialization.
- Sometimes it is necessary (and good for given application) to
break layers. For example, it is good for an HTTP agent to be aware
of TCP speeds and round trip times to different mirror servers in
order to optimize the choice of server. When designing
specification, identify the functionalities that break layers so it
is clear when they are being used.
The section on architectural
specifications includes references to additional format
specification guidelines.
Other design issues:
This section discusses important characteristics of data formats
which can together be used to describe and understand them.
A textual data format is one in which the data is specified as a
linear sequence of characters. HTML, Internet e-mail, and all
XML-based languages are textual. In modern textual data formats,
the characters are usually taken from the Unicode repertoire.
Binary data formats are those in which portions of the data are
encoded for direct use by computer processors, for example
thirty-two bit little-endian two's-complement and sixty-four bit
IEEE double-precision floating-point. The portions of data so
represented are include numeric values, pointers, and compressed
data of all sorts.
In principle, all data can be represented using textual
formats.
The trade-offs between binary and textual data formats are
complex and application-dependent. Binary formats can be
substantially more compact, particularly for complex pointer-rich
data structures. Also, they can be consumed more rapidly by
software in those cases where they can be loaded into memory and
used with little or no conversion.
Textual formats are often more portable and interoperable, since
there are fewer choices for representation of the basic units
(characters), and those choices are well-understood and widely
implemented.
Textual formats also have the considerable advantage that they
can be directly read and understood by human beings. This can
simplify the tasks of creating and maintaining processing software,
and allow the direct intervention of humans in the processing chain
without recourse to tools any more complex than the ubiquitous text
editor. Finally, it simplifies the necessary human task of learning
about new data formats (the "View Source" effect).
All things being equal (a rare state of affairs) textual formats
are generally preferable to binary ones in Web applications.
It is important to emphasize that intuition as to such matters
as data size and processing speed are not a reliable guide in data
format design; quantitative studies are essential to a correct
understanding of the trade-offs.
TAG issue binaryXML-30:
Effect of Mobile on architecture - size, complexity, memory
constraints. Binary infosets, storage efficiency.
Final-form data formats are not designed to allow modification
or uses other than that intended by their designers. An example
would be PDF, which is designed to support the presentation of page
images on either screen or paper, and is not readily used in any
other way. XSL Formatting Objects (XSL-FO) share this
characteristic.
XHTML, on the other hand, can be and is put to a variety of uses
including direct display (with highly flexible display semantics),
processing by network-sensitive Web spiders to support search and
retrieval operations, and reprocessing into a variety of derivative
forms.
In general XML-based data formats are more re-usable and
repurposable than the alternatives, although the example of XSL-FO
shows that this is not an absolute.
There are many cases where final-form is an application
requirement; representations which embody legally-binding
transactions are an obvious example. In such cases, the use of
digital signatures may be appropriate to achieve immutability,
whether the format is naturally final-form or some XML
vocabulary.
On the other hand, where such requirements are not in play,
representations that are reusable and repurposable are in general
higher in value, particularly in the case where the information's
utility may be long-lived.
Some data formats are explicitly designed to be used in
combination with others, while some are designed for standalone
use. An example of a standalone data format is PDF; it is not
typically embedded in representations encoded in other formats.
At the other extreme is SOAP, which is designed explicitly to
contain a "payload" in some non-SOAP vocabulary. Another example is
SVG, which is designed to be included in compound documents, and
which may in turn contain information encoded in other XML
vocabularies.
This characteristic is related to, but distinct from, the
final-form/reusable distinction discussed above. For example, one
can certainly imagine cases where it is useful for a representation
to include data in multiple different formats, but be considered
immutable and display-only.
TAG issue xmlProfiles-29:
When, whither and how to profile W3C specifications in the XML
Family?
TAG issue mixedUIXMLNamespace-33:
Composability for user interface-oriented XML namespaces
TAG issue xmlFunctions-34:
XML Transformation and composability (e.g., XSLT, XInclude,
Encryption)
TAG issue RDFinXHTML-35:
Syntax and semantics for embedding RDF in XHTML
- Versioning
- Backward compatibility
- Forward compatibility
- Optional v. mandatory extensions
- Schema restriction if designing for extensibility.
Deterministic/non-deterministic content models
More incoming from D. Orchard
Editor's note: Expect to add
reference Web
Architecture: Extensible Languages.
In many cases, the information contained in a separation is
logically separable from the choice of ways in which it may be
presented to a human, and the modes of interaction it may
support.
While such separation is, where possible, often advantageous, it
is clearly not always possible and in some cases not desirable
either.
More incoming from C. Lilley
One of the greatest strengths of HTML as a format is that it
allows authors to embed cross references (hyperlinks). The
simplicity of <a href="#foo"> as a link to "foo" and <a
name="foo"> as the anchor "foo" are partly (perhaps largely)
responsible for the birth of the hypertext Web as we know it
today.
Simple, single-ended, single-direction, inline links are not the
most powerful linking paradigm imaginable. But they are very easy
to understand. And they can be authored by individuals (or other
agents) that have no control or write access to the other end
point.
More sophisticated linking mechanisms have been invented for the
Web. XPointer allows links to address content that does not have an
explicit, named anchor. XLink allows links to have multiple ends
and to be expressed either inline or in "link bases" stored
external to any or all of the resources identified by the links it
contains.
All of the current common linking mechanisms identify resources
by URI and optionally identify portions (or views) of a resource
with the fragment identifier. The almost universal appeal of
linking between resources suggests that inventors of new
representation formats SHOULD provide mechanisms for identifying
links to other resources. Representation formats based on XML
SHOULD examine XPointer and XLink for inspiration.
The common need to point into a representation, that is, to
identify some portion of its content (or some view of its content)
besides the entire, monolithic representation suggests that
inventors of new representation formats SHOULD provide mechanisms
for identifying portions of their format. This can most often be
achieved by describing the fragment identifier syntax for the media
type that identifies their resource format. Representation formats
based on XML SHOULD use at least the XPointer Framework and
XPointer element() Schemes for their fragment identifier
syntax.
If a future revision of RFC 3023 identifies the XPointer
Framework, element(), and perhaps other ancillary schemes as the
fragment identifier syntax for XML documents, authors will be able
to rely on at least those schemes for all XML documents.
TAG issue: What is the scope of using XLink? xlinkScope-23.
Many resource representations are encoded in formats which are
XML vocabularies. This section discusses issues that are specific
to such data formats.
Anyone seeking guidance in this area is urged to consult the
"Guidelines For The Use of XML in IETF Protocols" [IETFXML] for the use of XML in Internet
Protocols. This document contains a very thorough discussion of the
considerations that govern whether or not XML ought to be used, as
well as specific guidelines on how it ought to be used. While it is
directed at Internet applications with specific reference to
protocols, the discussion is generally applicable to Web scenarios
as well.
The discussion here should be seen as ancillary to the content
of the IETF BCP. Refer also to "XML Accessibility Guidelines" [XAG] for help designing XML formats
that lower barriers to Web accessibility for people with
disabilities.
XML defines textual data formats that are naturally suited to
describing data objects which are hierarchical and processed in an
in-order sequence. It is widely but not universally applicable for
format specifications. For example, an audio or video format is
unlikely to be well suited to representation in XML. Design
constraints that would suggest the use of XML include:
- Explicit representation of a hierarchical structure.
- The data's usefulness should outlive the tools currently used
to process it.
- Ability to support internationalization in a self-describing
way that makes confusion over coding options unlikely.
- Early detection of encoding errors with no requirement to "work
around" such errors.
- A high proportion of human-readable textual content.
- Potential composition of the data format with other XML-encoded
formats.
Editor's note: Which XML
Specifications make up the XML Family?
The Web is significantly a networked information
system. Authors and applications can use URIs uniformly to identify
different resources on the Web. After representations of these
resources have been retrieved, they may be processed in a variety
of ways. Some applications (and some users) will undoubtedly build
new resources by combining several representations together. This
is particularly easy, and potentially useful, when XML
representations are available for all the resources.
However, combining representations in this way moves them out of
their original context and places them in a new context. This
change of context introduces the possibility of information loss.
Any information that depended on the local context will no longer
be available.
What is needed is a mechanism for establishing a global context
for the elements and attributes in the XML resources. This problem
bears a strong resemblance to the distinction between relative and
absolute URIs. While the many hundreds of relative URI references
to "index.html" on a typical web server may be entirely unambiguous
in their respective contexts, they have no unambiguous global
meaning. But each such relative URI has an unambiguous absolute URI
that can be established in its local context and used when a
document is moved. This solves the problem for URI references.
For elements and attributes, their names can be seen as
analogous to relative URI. Within their original context, they have
meanings that are clear and entirely unambiguous. Namespaces in XML
provides a mechanism for establishing a globally unique name that
can be understood in any context.
The "absolute" form of an XML element or attribute name is the
combination of its namespace URI and its local name. This is
represented lexically in documents by associating namespace names
with (optional) prefixes and combining prefixes and local names
with a colon as described in "Namespaces in XML" [XMLNS].
Designers that use namespaces are thus providing a global
context for documents authored with their schema. Establishing this
global context allows their documents (and portions of their
documents) to be reused and combined in novel ways not yet
imagined. Failure to provide a namespace makes such reuse more
difficult, perhaps impractical in some cases.
The most significant technical drawback to using namespaces is
that they do not interact well with DTDs. DTDs perform validation
based on the lexical form of the name, making prefixes semantically
significant in ways that are not desirable. As other schema
language technologies become widely deployed, this drawback will
diminish in significance.
Namespace designers SHOULD make available human-readable
material to meet the needs of those who will use the namespace
vocabulary. The simplest way to achieve this is for the namespace
name to be an HTTP URI which may be dereferenced to access this
material. The resource identified by such a URI is called a
"namespace document."
There are many reasons why a person or agent might want more
information about the namespace. A person might want to:
- understand its purpose,
- find out who controls it,
- request authority to access schemas or collateral material
about it, or
- report a bug or situation that could be considered an error in
some collateral material.
A namespace document should also support the automatic retrieval
of other Web resources that support the processing markup from this
vocabulary. Useful information to processors includes:
- schemas, to use for validation,
- style sheets, to use for presentation,
- ontologies, to use for making inferences, or
- any number of other application-specific details.
It follows that there is, in general, no single type of resource
that can be returned in response to a request for the namespace
name that will always be the most appropriate; see the section on
future work regarding namespace
document formats for more information.
Issue: namespaceDocument-8:
What should a "namespace document" look like?
Issue:
abstractComponentRefs-37: Definition of abstract components
with namespace names and frag ids
Editor's note: Where should we put a
section on mixing namespaces; is the section on processing model
more appropriate? See issue
mixedUIXMLNamespace-33.
Suppose that the URI http://example.com/oaxaca
defines a resource with representations encoded in XML. What, then,
is the interpretation of the URI
http://example.org/oaxaca#weather?
The URI specification [URI]
makes it clear that the interpretation depends on the context of
the media type of the representation. It follows from this that
designers of XML-based data formats SHOULD include the semantics of
fragment identifiers in their designs. The XPointer Framework [XPTRFR] provides a syntax designed
for in such fragment identifiers, and it SHOULD be used for this
purpose.
When a representation is provided whose media type is
application/xml, there are no semantics defined for
fragment identifiers, and thus they SHOULD NOT be provided for such
representations. This is also the case if the representation is
known to be XML because the media type has a suffix of
+xml as described in "XML Media Types" [RFC3023], but there is no
normative specification of fragment semantics.
It is common practice to assume that when an element has an
attribute that is declared in a DTD to be of type ID, then the
fragment identifier #abc identifies the element which
has an attribute of that type whose value is "abc".
However, there is no normative support for this assumption and it
is problematic in practice, since the only defined way to establish
that an attribute is of type ID is via a DTD, which may not exist
or may not be available.
TAG issue fragmentInXML-28:
Do fragment identifiers refer to a syntactic element (at least for
XML content), or can they refer to abstractions? See TAG issue.
TAG issue xmlIDSemantics-32:
How should the problem of identifying ID semantics in XML languages
be addressed in the absence of a DTD? See draft
TAG finding "How
should the problem of identifying ID semantics in XML languages be
addressed in the absence of a DTD?".
RFC 3023 defines the media types application/xml
and text/xml, and describes a convention whereby
XML-based data formats use media types with a +xml
suffix, for example image/svg+xml.
In general, media types beginning with text/ SHOULD
NOT be used for XML representations. They create two problems:
First, intermediate agents in the Web are allowed to "transcode",
i.e., convert one character encoding to another. Since XML
documents are designed to allow them to be self-describing, and
since this is a good and widely-followed practice, any such
transcoding will make the self-description false.
Secondly, representations whose media types begin with
text/ are required, unless the charset
parameter is specified, to be considered to be encoded in US-ASCII.
In the case of XML, since it is self-describing, it is good
practice to omit the charset parameter, and since XML
is very often not encoded in US-ASCII, the use of
"text/" media types effectively precludes this good
practice.
The Resource Directory Description Language [
RDDL] is a proposal under discussion in the
community for a variant of XHTML optimized for the construction of
namespace documents which meet the goals described in this section.
Note, however, that RDDL or a document like it, is no more
universally correct than any other type of representation.
Namespace developers should give careful consideration to choosing
the most appropriate format for their application, keeping in mind
that both human- and machine-readable information is useful.
