1Handling of file:// URIs:
2
3This directory contains code to map basic boto connection, bucket, and key
4operations onto files in the local filesystem, in support of file://
5URI operations.
6
7Bucket storage operations cannot be mapped completely onto a file system
8because of the different naming semantics in these types of systems: the
9former have a flat name space of objects within each named bucket; the
10latter have a hierarchical name space of files, and nothing corresponding to
11the notion of a bucket. The mapping we selected was guided by the desire
12to achieve meaningful semantics for a useful subset of operations that can
13be implemented polymorphically across both types of systems. We considered
14several possibilities for mapping path names to bucket + object name:
15
161) bucket = the file system root or local directory (for absolute vs
17relative file:// URIs, respectively) and object = remainder of path.
18We discarded this choice because the get_all_keys() method doesn't make
19sense under this approach: Enumerating all files under the root or current
20directory could include more than the caller intended. For example,
21StorageUri("file:///usr/bin/X11/vim").get_all_keys() would enumerate all
22files in the file system.
23
242) bucket is treated mostly as an anonymous placeholder, with the object
25name holding the URI path (minus the "file://" part). Two sub-options,
26for object enumeration (the get_all_keys() call):
27  a) disallow get_all_keys(). This isn't great, as then the caller must
28     know the URI type before deciding whether to make this call.
29  b) return the single key for which this "bucket" was defined.
30     Note that this option means the app cannot use this API for listing
31     contents of the file system. While that makes the API less generally
32     useful, it avoids the potentially dangerous/unintended consequences
33     noted in option (1) above.
34
35We selected 2b, resulting in a class hierarchy where StorageUri is an abstract
36class, with FileStorageUri and BucketStorageUri subclasses.
37
38Some additional notes:
39
40BucketStorageUri and FileStorageUri each implement these methods:
41  - clone_replace_name() creates a same-type URI with a
42    different object name - which is useful for various enumeration cases
43    (e.g., implementing wildcarding in a command line utility).
44  - names_container() determines if the given URI names a container for
45    multiple objects/files - i.e., a bucket or directory.
46  - names_singleton() determines if the given URI names an individual object
47    or file.
48  - is_file_uri() and is_cloud_uri() determine if the given URI is a
49    FileStorageUri or BucketStorageUri, respectively
50