Python's Hardest Problem

jknupp · 2 months

Can't this trivially be said of any NoSQL implementation? The point of this was to optimize on space while still keeping objects in memory for fast lookups. How exactly would a traditional database help here? Can I expect PostgreSQL to perform my string compression optimizations?

jknupp · 2 months

Sorry, the real point of this was to remove raw pointers so that in the future multiple instances of the db could operate on the same segment simultaneously. An added benefit was the removal of a redundant vector of pointers, saving the sizeof a pointer per record.

jknupp · 2 months

Sorry, does the MS_ASYNC flag to msync not write pages asynchronously?

jknupp · 2 months

Fixed. Thanks.

jknupp · 2 months

I actually don't even use a vector or pointers anymore (since an hour ago). The integer key is used to compute an offset into shared memory (since I know the address that was mapped at). This means I don't need the overhead of ANY data structure to store the values.

jknupp · 2 months

std::vector. Since my keys are unique monotonically increasing integers, I don't have to worry about using a "real" hash table. I just have a vector of pointers.

jknupp · 2 months

I'm not doing the lookup of the key in Python. I'm doing a lookup into a C++ hash table with monotonically increasing integer keys, which is certainly O(1). surpdb is written in C++ with Python bindings.

jknupp · 2 months

I certainly am, but it's very specific to my application at the moment. I'm writing some code generators to make it less-so, but this will take a bit of time. I hope to have it up on Github sometime this week.

jknupp · 3 months

Well, some people want the fastest Porsche possible and are willing to do some creative stuff to make that happen.

jknupp · 3 months

Because the boost python library is much easier to work with. I've used the C api before (as I stated in my post) and it's a hassle. That's why boost::python even exists.

jknupp · 3 months

I agree it's more interesting for some, but probably a very small portion of the overall Django user-base. I posted that entry over in /r/django so we'll see. As an aside, I didn't mean for the topic to be linkbaity, it's just part of a series of articles I'm writing on professional-grade Django development.

jknupp · 3 months

That was the answer this time, but not always. For hard-core optimizations, see my most recent post on optimizing Django views using C++: http://www.jeffknupp.com/blog/2012/02/15/optimizing-django-views-with-c-plus-plus/

jknupp · 3 months

Well, when I was new to Django in creating my first site, I followed exactly none of these steps. I wish someone had told me then what's on the blog. It's not a post meant for seasoned Django programmers. Rather, it's aimed at those new to Django looking to create their first production-quality site.

jknupp · 3 months

Thanks, I'll check them out. Always looking for new tools.

jknupp · 3 months

I'd say that this approach is less susceptible to feature creep then a bottom-up approach, as you know for a fact what features you need. That said, you can't go crazy when writing the client code. It should be clear and easy to use, but that doesn't necessarily mean loaded with features.

Two-Year Club	reddit gold Since January 2012
Verified Email

jknupp

TROPHY CASE

you'll need to login or register to do that

create a new account

login