HN Explorer

21 comments

[ 3.2 ms ] story [ 18.2 ms ] thread
amyjess
Oh, this looks really nice. Thank you for posting it!

A while back, I had an idea for modelling a filesystem as a dict, so maybe this will inspire me to get off my ass and actually write it.

Basically, the idea is that this:

    foo = rootfs['home']['amyjess']['stuff']['foo.txt']
    rootfs['home']['amyjess']['stuff']['bar.txt'] = bar
would be equivalent to this:

    with open('/home/amyjess/stuff/foo.txt', 'r') as foo_file:
        foo = foo_file.read()
    with open('/home/amyjess/stuff/bar.txt', 'w') as bar_file:
        bar_file.write(bar)
I really want to implement that now.
billowycoat
Nice idea!

You can also do this with PyFilesystem...

    fs.gettext('/home/amyjess/stuff/foo.txt')
    fs.settext('/home/amyjess/stuff/foo.txt', bar)
There are equivalent bytes methods.
dom0
You might want to look at pathlib:

    with (Path.home() / '.config' / '1234.conf').open() as fd:
        print('Hello', file=fd)
Granted, with / precedence you'll often need parens to group / before .
existencebox
It's funny you mention that, I have literally that functionality in my "blob storage" python wrapper. Currently it's just like 50 lines of code I use for internal ad-hoc storage because as it turns out, reasoning about storage as a hierarchy is really nice. (I also let you index with a list, where the list is the hierarchical path; the key part being composibility, which I imagine you can do reasonably with both models).

Interesting to know that it's a more common pattern. I will have to think a bit on where that may go. I had always shot down any "value" with "it's just a thin transformation on top of path.join". (yay self-deprecation or something?)

m45t3r
There is already pathlib in Python's stdlib that do something similar:

>>> p = PurePath('/etc') >>> p PurePosixPath('/etc') >>> p / 'init.d' / 'apache2' PurePosixPath('/etc/init.d/apache2')

And yeah, this uses operator overload. However it does look quite nice.

gumby
You might want to look at how Common Lisp abstracted the filesystem 20-odd years ago as an example of how to do it right and also how to do it wrong (e.g. lack of iterators in those days)
straphka
I once made something like this to mount json files as a fuse filesystem. It's not maintained and one of my first projects, so the code is somewhat questionable, but it does work.

It basically mounts a dict on a filesystem, the exact opposite of what you want :)

https://github.com/yhekma/datamounter

ekiara
My big question is why? you wanted to navigate or display a JSON dataset using filesystem-viewer tools? i.e. "Explore" the JSON dataset?
straphka
Well, mostly because I wanted to play with fuse, but it started out as an ansible thing (doesn't work with >2 though).

The ansible setup module returns the system information in json, and if you mount it with the --realtime flag, when you open a "file" like ram for instance, ansible fetches the current value for you. That way you have your infrastructure mounted so to speak. Sort of a /proc filesystem for your linux infrastructure.

I never got further than a working poc though. It works, but there are some bugs and there is no regard for security.

billowycoat
Come to think of it, I have already implemented something like this in Moya.

This would insert "bar/baz.txt" from filesystem "foo" in to a template:

    ${.fs.foo.bar.baz.txt}
O5vYtytb
pyfilesystem is great!

My only suggestion would be to improve the search optimization for pypi.

It's the ~10th result for https://pypi.python.org/pypi?%3Aaction=search&term=pyfilesys... and a page down on https://pypi.python.org/pypi?%3Aaction=search&term=fs&submit...

It's really not clear what it's called since the package name is 'fs' but the project is 'pyfilesystem'.

billowycoat
> pyfilesystem is great!

Cheers.

> My only suggestion would be to improve the search optimization for pypi.

Fair point. Not entirely sure how to fix that. I'll look in to it...

linsomniac
PKG-INFO, line 2:

Name: fs

Looks like it's also registered in PyPI as "fs", from the URL: https://pypi.python.org/pypi/fs/

adtac
Maybe have a proxy https://pypi.python.org/pypi/pyfilesystem that has `fs` as a requirement? That way even if people install with `pip install pyfilesystem`, you're good.
billowycoat
Hmm. That's an idea. Feels like a hack though. Do you know if there is a precedence for that?
adtac
We're planning on something similar to move a big open source library somewhere else (so that dependencies don't break).
xwvvvvwx
If you run pip install py.test it fails with a message that you should run pip install pytest.
j_s
As long as you can recursively process folders and files without stopping early due to an UnauthorizedAccessException-equivalent (which .NET didn't straighten out 'til v4).

http://stackoverflow.com/a/2663779

teilo
Would this work with any object implementing the PathLike interface introduced in Python 3.6 ?