One thing I've thought of is that on android, it's not so hard to get app usage in a custom app, as a proxy for being on the phone. So it could check if you've been on your phone for 15+ minutes and then ping you only then. It would take some app -> server -> telegram app communication
Wonder if you could do it with Flutter. For me it's a bit of an investment to learn Kotlin/Swift just for this. Tried in the past Android is a beast imo but my opinion is low as I'm primarily a web dev.
Author here: Yes, exactly which is why I chose to use Telegram to enter my data. If you go all the way to the bottom of howisFelix.today and check out the "Data Inputs" section I wrote a little about it. Being able to easily answer on whatever device you're on right now makes everything easier
It's a bit heavier on the automatic data aggregation side, but has some manual inputs as data sources too.
I'm hoping to make it easier to interface with existing tools for data analysis, dashboards, notebooks etc.
https://beepb00p.xyz/myinfra.html is, to me, the definitive go to "look how ultra we could be" site of yours, @karlicoss. i show this off to people on the regular, it's so impressive.
i feel like there's such a gap, that it takes such dedicatiom & effort to have the digital world we interact with report back & give us the facts of whats occured. i dont know what futures this obstructs but it's a drain on general human possobility & energy that man-made reality, unlike the gods, is so deeply resistant to revelation & exploration.
i've seen comments calling the submission a vanity project, and it admits to it's owm uninterestingness to a certain degree. i think karlicoss's far broader spanning digital self sensing & ordering has a far higher chamce of helping us distill out meaning & sense from the world. but what a bold & hard sojourn just to start, just to begin to marshal the data. critics abound but i for one think we have so much, are only just starting to understand & calobrate ourselves in a technoaugmemted mode.
currently the technology all resists technoaugmentative exploration. this is a crime against humanity, against our base human spirit. we dont know what is possible, what we could do. we cannot explore better possibilities. we have now two examples of goong far, but all the systems around us resist this open exploration: we are all chained to the wall of plato's cave, and few bother to break free. i await & pursue an emergence.
merely trying to give the rest of the universe a little better chance at connecting & attaining excellence. it's you, @karlicoss, advancing us all, if any of us are at all hip to listen & recognize. my sincere thanks, you are an inspiration.
I think bots are a great way of automating your life in fun ways. It's also my new favorite project to suggest to people starting to program, because just a little bit of customizability goes a long way
Tracking mood is great. A simple telegram bot -> a google sheet lets telegram do the mobile/desktop UI and you just control the logic
One overlooked thing is automating financial tracking/categorization. In the US there's no good cross banking api except for email. Almost every credit card, bank, etc can email a receipt (set the minimum amount of alerts to be $0.01). The flowchart for parsing is bank alert -> gmail -> gmail filter by from address adds a label -> google pub sub notices that label and pings a little VPS somewhere -> gmail api -> parse email -> telegram bot
As I type, it sounds complicated but each step is well trod and has tutorials available. Now you have the ability to make custom push notifications for any transaction with < 1 second latency, instead of using the bank app. And it's all under your control. So the telegram bot allows you to categorize those purchases, you can add little notes or put it in categories. The nice thing is instead of looking at the end of the month and having 100+ little purchases and trying to remember if you ate at Starbucks 4 weeks ago, you categorize things as they happen in just a few clicks. And adding a category (one click) at the time implicitly marks the purchase as 'known good', not a suspicious purchase. So you can cross check at the end of the month for purchases you didn't categorize as suspicious.
If you own the server, it might be easier to set up a little smtp client. Or to be more security conscious, a incoming only smtp client like https://github.com/0xERR0R/mailcatcher/ and collapse that dataflow. But pubsub has a monstrous free tier anyway and I also wanted to keep a copy of the emails on gmail which is easier to search
Right that makes sense. It’s not a big deal. An app I was using MoneyWiz, let you switch between providers so something like Venmo or crappy store branded credit cards were able to be included too via Yodlee. I dont do much crypto, but Salt Edge does help there.
In case you don't make it through to the project's webpage howisfelix.today, and read through to the end, here is the creator's conclusion:
Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
I’ll likely continue tracking my mood, as well as a few other key metrics, however will significantly reduce the amount of time I invest in it.
> There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
This is my general feeling towards all quantified-self projects. Perhaps it creates interesting insights for some people, but I can imagine the kinds of conclusions I'd actually get from it will be things like "I'm happier on sunny days" and I'll say "yeah..."
It's a bit more nuanced. It's true that you do research looking for insights that would give you a competitive advantage, and you don't find many - and very often the results don't justify the cost. On the other hand, if you don't do that, you risk staying behind, so some form of data analysis is absolutely necessary for any business - you can't just run on your gut feeling especially once you exceed a certain size.
As a side note, people tend to be very unimaginative in the initial stages of data analysis: very often they just take the data they have instead of thinking what data they need to have. And obviously the results correspond to the initial limitations. So much effort is put into data munging and other stages when your inputs are incomplete or plain irrelevant to what your business actually needs.
Data Science manager here. I've worked with companies that didn't know the profitability of their different client contracts because the data was stored in separate systems. Fixing that kind of thing is super profitable. Every additional data project after that adds less incremental value, and it's easy to hit the point where more analysis doesn't make enough money to be worth it.
Maybe the executive is asking the wrong questions and puts it on the data scientist. Hint : 'Can you create a report that shows that my last decision was a success ?' is not a good question.
It can be more useful in some cases. E.g. my Fitbit does get me to move more when I notice I've done too few steps and tracking mood, productivity etc. is much more useful when you actually try things like nootropics to actually figure out if they seem to help you.
It doesn't make a lot of sense to try to correlate things with your mood that you think are correlated. Because giving a mood grade is biased. For example if you feel like you don't sleep enough, the day you will sleep a lot you will want to give a higher grade.
For me "quantified self" has been half meditative. I have tracked daily water intake (and only water to keep it simple) and that has been a huge noticeable quality of life improvement. Waterminder on apple watch is the best.
This year I added in daily mood tracking, and it's mostly focused on taking 5 minutes a day for myself to just jot notes down quickly. I'm using dailo and I have a few categories too (work good/ok/bad - food homecooked/takeout - vices) and just the space I've created to do that has been worth it. I don't expect to get anything too interesting long term besides patterns in takeout/cooking, but I find myself more connected to my day good or bad, and that presence of mind to combat the autopilot is worth it.
What was the quality of life improvement in tracking water intake? I find it interesting, because your body tracks it whether you want to or not, and I never found forcing myself to drink more water beneficial.
It’s surprisingly easy for some to get disconnected from the signs of dehydration. For example: when focused on a project I can go a couple days before realizing that I did not drink any water at all.
All that dehydration talk is fascinating to me. It’s like most people are aliens. I drink about 6L per day just in tap water, not including coffee and tea. I love drinking water and could not imagine limiting myself to what others perceive as their daily goal. And as the question tends to come up: I’m not diabetic, and I have a home-office job. I do vape.
The signs I easily ignore are a little bit of brain fog, some reduction in physical energy, and dry skin. The dry skin is meh, and the other two are solved* with food that provides energy.
* = It does not actually resolve it, but it does “enough” and has a long enough lead time that it’s easy to have an instinct of “Oh I’m feeling run down, time to grab a snack” followed by “that didn’t quite get where I expected, I’ll give it time to digest” before forgetting about the whole thing as the project reabsorbs my attention.
I briefly tracked water intake since it was a feature in the Garmin app I was using for activity tracking. Ironically it made me worse off, because it interrupted my (extremely simple) natural hydration process.
There were 3 customizable preset capacities, but even using those required a couple taps each time. Drinking a non-standard amount and manually adjust it required extra effort and taps. Sometimes I’d forget to record it right away, then have to try to remember if I was double-counting a refill. The mental burden was clearly out of balance with the (practically nonexistent) reward.
I quit tracking it when I found myself adjusting my behavior, like avoiding the natural habit to top off my water bottle when it was less than half full and I was getting up from the desk anyway. Ultimately it would be interesting to have that data, but it’s only worth it to me for zero cost/effort.
I think for me it helped me to see how often when I was thirsty I might actually eat a snack.
From there, it's also valuable because it's clear that in the past I might have a dehydration headache, eat something, and still have a headache because I was thirsty. By just incorporating a routine and simple 1 click tracking, that consistency has helped me to easily manage this. But as you said, for you it works "naturally". For me, I was pretty out of tune with that.
It's still useful for uploading in a couple of decades. The more digital crumbs we leave, the more will be uploaded. Has anyone tried to fine-tune GPT-3 on their own comments?
As a joke/experimental project, I trained gpt-2 (don't have access to 3) on Joe Biden speeches/interviews/press conferences, cloned his voice (mumbling and all), created a 3d model of him, and connected Twitch chat to make it interactive for the audience.
Didn't do it for politics, just for lolz and prototyping a concept. Got to Twitch's advertising unlock, but stopped running it because it was a headache to maintain with my home setups/remotely.
But yeah, not trained on my comments, but someone else's, and it didn't take a ton of content to get it going for gpt-2.
I would be interested in detection and correlation abilities here. I think simply collecting the data and graphing it doesn't help.
However, if enough metrics were automatically collected (when did I fall asleep and heart rate lower?) and there was some system that could sort through them (eating a 4th meal increases odds of ruby team scoring additional points by a factor of 1.3x) then you could possibly learn some neat things.
Some of the most interesting things have been pointed out to me by others. Things that are related but I didn't notice. Foods that seem to increase my chances of sickness and things like that.
When reading above the experience I have is that I rapidly correlated with apple watch heart data that my heart rate stays high when I drink alcohol in the evening, resulting in poor sleep.
Conversely I can see when I have been likely to have been drinking from my heart rate.
I think the key here is the passive data collection with intuitive data presentation to drive low barrier to entry analytics.
Fun fact: British Petroleum coined the term “carbon footprint”. It had gotten to the point where it couldn’t deny climate change. So it was time to get well-meaning middle class folks to blame each other instead of the petroleum industry. (See also in recent years the obsession with plastic straws and plastic bags.)
Agreed, the metrics he uses for CO2 consumption via flying are misleading: he compares the flying against total average CO2 production per US Citizen per year. Clearly, he's doing more than just flying in his life, so the comparison is meaningless.
I don't feel you will be able to detect big correlation by using that kind of system. If there is a big correlation to be found, it is likely to have already been found by researchers.
I'm tracking my life by a similar system than OP, but the error with tracking is to start tracking useless things. For example OP track his calorie intake, it is a lot of work, and it is not really usefull at the end. It is unlikely to correlate with any other measures he is doing (apart from weight).
I personally only track mood, activities and position. Each tracking has a specific goal, and only take me 1 minute per day. I have a lot of headaches and could be tempted to track headaches to understand what triggers them. But now that I know what triggers them (when I'm hungry, or I spent too much time exposed to the sun) I would never have been able to detect that correlation from data.
So my advice to people wanting to track their life, identify first what you need the data for, and increase the number of data you track if you have a lack of data to achieve a specific goal, but don't track for the sake of tracking.
For example I track my position, to know where I have been, but also to warn my parents in case I go missing and nobody know where is my last position
I track my mood, to know if I am doing better over time, or if I'm able to improve my sleep not only by feeling it, but by having numbers on
I track my activitites, because I want to know If I work too much, if I do enough sport, or if I spend too much time on my computer in a week
>> For example OP track his calorie intake, it is a lot of work, and it is not really usefull at the end. It is unlikely to correlate with any other measures he is doing (apart from weight).
Tracking calories is very very simple if you use an app like MyFitnessPal. For most foods you can scan the barcode or you can just enter the calorie data from the label.
Secondly, isn't weight a pretty important measure to track? It can impact a lot of things and therefore if you see you start to have too many days in a week that are above your calorie goal you can catch any gain before noticing it on the scales.
Author here - Tracking calories has impact on way more things than just the weight. e.g. choosing a low-fat diet vs a low-carb diet makes you feel very differently day to day, and every person might prefer something else. Also, there is a huge difference in energy levels and quality of sleep while I'm cutting vs while I'm gaining weight.
The problem is waiting for three years to gain insight. This non-lean approach fails because in retrospective analyses context is lost, there's hindsight and survivorship biases. The opportunity to discover more relevant data sources is missed.
I also feel like it gets to a roundaboutway of 'proving' things people already know. More sleep is nicer, unless it's so long it's bc you're sick. Drinking more means more dancing. Days when you sit all day working are less fun than walking around with friends (and, of course, not working then)
I feel like keeping a rough journal and manually revisiting would be more helpful than quantifying and correlating. But financial tracking can be useful, to a smaller degree, and tracking mood changes over big life changes can be valuable I guess. Otherwise, does feel vanity
True, but it seems like it would be nice to know where to draw the lines. More sleep is good, but how much more? Etc.
Sometimes, it feels like there's so much going on that it's hard to know if it was the sleep, the diet, the events, or something else. That's where the data might be nice.
If you read to the bottom of the site, you'll find that the author agrees with you that the end result was not particularly useful:
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
But calling it a vanity project seems overly dismissive. Sometimes you start a project and don't know how it's going to end up. Sometimes it's the journey and not the destination that makes a project worthwhile. I think Felix deserves a lot of credit for writing up his conclusions and sharing them.
I think the main value in the "quantified self" area is (1) when there's something wrong and you're working to fix it (food allergies, mood disorders), and (2) insights gained by comparing data from millions of users.
Neat, and interesting that his conclusion is that it's too much effort to do for oneself.
My take on this is Timelinize[0], which is capable of downloading all your data from various digital records like social media, your phone, arbitrary files and unstructured folders, your Google account, etc. So it can aggregate data from Google Photos, Location History, Twitter, Facebook, text messages, and more, onto your computer, organized into a single timeline.
(It is the successor to Timeliner[1].)
I suppose Timelinize can support personal body metrics from various devices, and it is planned to support manual entries as well like journals. Analyses like his would also be possible.
Data can be automatically imported on a regular basis via APIs (like, backing up your new tweets every day). The arbitrary file importer can also be scheduled to watch a directory for changes: just write a file or folder to that directory, and Timelinize will automatically import it (and optionally delete it when finished).
Timelines are stored locally on your own computer, as plain files on disk, with a sqlite database as an index, so it should be a breeze to back up and use/read without proprietary software.
The idea is to:
- Back up your precious data that is otherwise held more-or-less hostage on somebody else's computers
- While creating a detailed, personal journal for your family history
- In a way that can show you your life in context with others' timelines and the wider world.
Great! First releases will probably be private betas.
Twitter yes, Facebook somewhat (for now -- the API and data graph are very complicated!), Reddit probably not initially (but on my lists; need to learn how to get the data), and Hacker News probably eventually (also need to learn how to get the data).
Still deciding; I want it to appeal to general audiences, rather than only techies. Timelinize is taking a lot of my free time (along with a new family life, etc.), so Timeliner isn't being updated right now.
(Also, I'm in the market for a better, available name.)
Thanks for sharing Timelinize and Timeliner. It's always great to meet other people in this space.
My own take on this problem is Chronomize [0], which I've been working on since 2020. It's interesting how we independently came up with similar names.
Feel free to reach out if you'd like to chat about personal data management.
Ooh, other people in this space - I remember seeing Timeliner a while ago while researching UIs for the backend I had built before starting on my own frontend [0]. Chronomize looks interesting (I used the macOS calendar for a while before outgrowing it, but still mostly maintain compatibility), but I'm curious how you'll handle certain UI things (like showing financial transactions, which generally have a date but not a time, or things like photo albums where I could mark off that an album existed but not link to any images within it).
What a great timing. I've recently started a side project to do something similar. It's todo/habit/tracking app made to scratch my own itches.
1. I've tried out many todo apps and they all aren't quite right for me (atm it's just a pen and paper), so I want to try design UX that's tailor made for me.
2. I found that arbitrary points and day streaks go a long way in making my habits stick. So I want to have an app that can add points/ streaks to anything that I do. (e.g. I complete "go to the gym" from my todo list, and it automatically adds a log to a table. Then that log is parsed to show some arbitrary point & streaks for going to the gym)
3. I want the app to have a REST endpoint that I can use to integrate with other apps to automate data entry.
Honestly, it's more of a procrastination thing than a productivity thing :p but have a lot of fun thinking about/ building the app.
This is excellent and hugely appreciate that it’s open source. My previous startup attempt was a self-hosted personal metrics app so similar to this. Instead I pivoted to making self-hosting web apps easier: will be adding a template for this asap.
My wife tracks just about everything in her life - and me almost nothing. I’d like the narrow the gap just slightly!
You know, if you’re really good with Postgres queries, you could run entire businesses, blogs, apps, just by using SQL queries and not even have a frontend or backend. And when you want to present your data, you could use a well formatted PostgreSQL view to make it easier to read. This man’s entire status page could just be a view with appropriate columns. And if you want others to read or write data, you could assign them roles with permissions so they don’t go screwing things up or seeing things they shouldn’t.
When I was looking at the stats, I realized I can never make something like this. Just alone the list of flying spots mentioned are huge for me. I don't think I have travelled at much internationally. Having a location graph for myself would be mostly few cities and maybe 1 or 2 countries(connecting flights).
"Joining Megaco also means you join our corporate wellness program, via which we track your important life parameters and help you achieve Maximum Happiness and optimise Life Satisfaction.
* Please note that a Happiness score of at least 70% must be maintained to retain employment. Additionally, you must think about Megaco in a positive way for at least 2% of your out-of-work hours."
All bleak jokes aside, interesting work by the author.
This is super cool, but it is all manual data entry. I've tried manual data entry for tracking my personal metrics and the gaps make the data relatively useless. Consistency is key.
Automation is key for 99% of the data with only a small amount requiring manual entry.
Author here. Yes, I agree. For this project, only 5% of the 383,000 data points were collected manually, the rest came from various data sources like RescueTime, Apple Health, Withings smart watch and such
Have you developed standardized automation and ETL integration patterns for pulling this data? What's your process for adding new automated data sources?
Author here - I actually talked about this at the very bottom of the howisFelix.today page, in the "Conclusion" section:
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
This is a really cool project, but I have to ask a very simple question: is there an immeasurable impact of trying to measure all the impacts on your life?
I collected a lot of data on my life, and tried to analyse it.
The learnings that I got out of it was very non-surprising. Basically, if I slept 8h and I worked out regularly, I was happier and had more energy. That was basically the only two things that really stood out amongst all the factors I tracked.
I've tried various schemes for measuring my life and what I found was most effective was keeping a simple emotional journal every night. I've fallen off the wagon, but what I found was that the process of doing a nightly retrospective of my day improved my mood, made me less likely to get angry and made it easier to stay focused.
This project is super cool. I would recommend the author starts tracking the amount of CO2 he emits in a year, which could easily be approximated with the flight/transportation and food data he has. Carbon emissions per Capita per year in the US: 18 metric tonnes. In Italy: 8 metric tonnes [1]. Author is probably close to 60-80 with all the transatlantic flights.
I'm happy to return to the comment section here and see I'm not alone in my frustration over the author seeming to take pride in this crazy amount of air miles. Until that chart I really enjoyed scrolling through the page, seeing all the time and obsession the author put into this.
Why get frustrated over something like this? You’re seeing one data point on how the author lives their life. I see these kinds of comments more and more and can’t understand the need to judge someone else, without full information, based on your own personal standards.
You're right, there could be many reasons. But given the time we live in know it seems like something you don't want to wear with pride or without a disclaimer.
It's a fine balance between caring about stuff and wanting to raise awareness while also seeing other human beings for who they are, assuming good intent.
>> But given the time we live in know it seems like something you don't want to wear with pride or without a disclaimer
I disagree (although 'I fly lots' would be a strange thing to ever be 'proud' of imo, regardless of the climate issues). While we should all be mindful of our individual impact, asking people to restrict themselves personally (or shaming them) on such an important and positive freedom as being able to travel widely just sits wrongly with me.
There are much lower hanging fruit as regards climate change to tackle first at a much higher level (e.g. government, multi-national business). I feel our focus should be on that. If you want to make personal sacrifices that's noble but shaming others for not making the same sacrifices is petty and makes it seem like your reasons for making the sacrifices are for appearance rather than because you care. NB: I mean this more generally, it's not wholly directed at your specific comment which wasn't super judgemental.
I partially agree with that shaming strangers might not be a good way forward. But I also believe in some basic level of 'holding each other accountable'.
Maybe, besides political leaders and businesses, that should be limited to inner circle family and friends with whom you have a more complex relationship, and where there is a little bit more receptiveness towards critique. Even in that context it's probably better to just live the lifestyle yourself and hope they will follow.
On the other hand, if you put something out there, to be discussed publicly on HN, I don't feel obliged to only give positive feedback necessarily. Also, the author picked this up in a really nice way and has already added carbon stats to the page.
But man, it's so hard if you deeply care about something to keep your mouth shut. I don't know if you have any topics that make you feel like that?
Edit: actually responding to the rest of your comment, I'm a strong believer in the power of individual choices. The whole plant-based movement for example in The Netherlands (and outside) over the last few years was fully driven by individual consumer choices and little bits of money shifting in other directions. European politics actually tried to hinder it for a while, but at some point when something becomes the popular thing to think, politics will have to follow.
>> But I also believe in some basic level of 'holding each other accountable'.
I agree. There are certainly climate-related things I try to hold my extended family to account over (e.g. basics like recycling). I guess where we disagree is where the line is and how much criticism we’re willing to accept and impart :)
Is it possible that this bothers you personally? Flying really is one of the most negatively impactful things that an individual can do. Shifting the focus to government or business may be 'also correct', but you could also call it whataboutism.
> You're right, there could be many reasons. But given the time we live in know it seems like something you don't want to wear with pride or without a disclaimer.
In other words: I am not saying that he should be ashamed of being a pollutant. But I am wondering why he isn’t ashamed of it.
Going through his blog, it appears the man is a digital nomad (and a minimalist in theory — he only owns two suitcases of stuff — which is important considering he's likely a wealthy person), and lives the life he loves; give him a break.
Btw, he isn't traveling just for the sake of traveling — he has conferences around the world, and he's a European working in USA which means he probably flies back and forth from his home country a lot more than an average American. (Doesn't help that North America is far away from everywhere else).
For what it's worth, he makes an attempt to schedule all his conferences such that he wouldn't have to keep flying to/from North America in between conferences.
Felix Krause, keep doing whatever it is that you do and enjoy your life.
Really great that you shared all that data and even added additional graphs, thanks for the inspiration. I think there were some recommendations that floated around that suggested we should be at 2 metric tonnes per person by 2050 to be able to maintain +2 C°.
For the more lazy among us, there's also Wren which lets you do some estimates: https://www.wren.co/
Then, they actually let you donate/spend a bit of money towards offsetting your individual carbon footprint, towards whatever initiatives you choose (personally donated towards planting trees).
It's not like it would have as much of a profound impact as stopping the mega-corps would, but it's still something and pretty easy to do.
Disclaimer: not affiliated with them in any way, just thought it's a cool idea.
> How much does your household spend on furniture and appliances each month?
A little = 140€ / month. Wow. What do people do, redecorate every year?
edit:
11 tons of CO₂ per year
That's 64% less than the Germany average for your household size.
I guess the two of us are pretty ecological ;) But I just realized they didn’t ask about heating (afaik oil based and I have only costs, no usage numbers), so that might make some difference.
Author here = I actually researched & published many privacy projects (https://krausefx.com/privacy), and deeply care about being in control over your data, and being explicit about what you choose to share.
- The data is stored in my own, private database, not connected to any service
- Much of the data visualised on that page, is data many larger companies already have about you. I figured, why wouldn't I want to benefit also?
- The data shown above seems very personal but doesn’t actually expose any real sensitive information. For example, disclosing your current location, your home address, or stores and places you frequently visits is sensitive and potentially dangerous. Even looking at the map of my movement data on my page, you won’t get any more information that what most people have in their public Twitter bio.
I made sure all graphs are non-interactive, and had all personal information stripped out.
174 comments
[ 3.1 ms ] story [ 227 ms ] threadHow do you ensure that you consistently track? Automated pings? Physical buttons/sensors at your desk or I guess computer uptime maybe.
Other thing is did you have any concerns about making this info public, I guess it's not locating you exactly.
It's a bit heavier on the automatic data aggregation side, but has some manual inputs as data sources too. I'm hoping to make it easier to interface with existing tools for data analysis, dashboards, notebooks etc.
i feel like there's such a gap, that it takes such dedicatiom & effort to have the digital world we interact with report back & give us the facts of whats occured. i dont know what futures this obstructs but it's a drain on general human possobility & energy that man-made reality, unlike the gods, is so deeply resistant to revelation & exploration.
i've seen comments calling the submission a vanity project, and it admits to it's owm uninterestingness to a certain degree. i think karlicoss's far broader spanning digital self sensing & ordering has a far higher chamce of helping us distill out meaning & sense from the world. but what a bold & hard sojourn just to start, just to begin to marshal the data. critics abound but i for one think we have so much, are only just starting to understand & calobrate ourselves in a technoaugmemted mode.
currently the technology all resists technoaugmentative exploration. this is a crime against humanity, against our base human spirit. we dont know what is possible, what we could do. we cannot explore better possibilities. we have now two examples of goong far, but all the systems around us resist this open exploration: we are all chained to the wall of plato's cave, and few bother to break free. i await & pursue an emergence.
Tracking mood is great. A simple telegram bot -> a google sheet lets telegram do the mobile/desktop UI and you just control the logic
One overlooked thing is automating financial tracking/categorization. In the US there's no good cross banking api except for email. Almost every credit card, bank, etc can email a receipt (set the minimum amount of alerts to be $0.01). The flowchart for parsing is bank alert -> gmail -> gmail filter by from address adds a label -> google pub sub notices that label and pings a little VPS somewhere -> gmail api -> parse email -> telegram bot
As I type, it sounds complicated but each step is well trod and has tutorials available. Now you have the ability to make custom push notifications for any transaction with < 1 second latency, instead of using the bank app. And it's all under your control. So the telegram bot allows you to categorize those purchases, you can add little notes or put it in categories. The nice thing is instead of looking at the end of the month and having 100+ little purchases and trying to remember if you ate at Starbucks 4 weeks ago, you categorize things as they happen in just a few clicks. And adding a category (one click) at the time implicitly marks the purchase as 'known good', not a suspicious purchase. So you can cross check at the end of the month for purchases you didn't categorize as suspicious.
https://toshl.com/bank-connections/
In case you don't make it through to the project's webpage howisfelix.today, and read through to the end, here is the creator's conclusion:
Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
I’ll likely continue tracking my mood, as well as a few other key metrics, however will significantly reduce the amount of time I invest in it.
This is my general feeling towards all quantified-self projects. Perhaps it creates interesting insights for some people, but I can imagine the kinds of conclusions I'd actually get from it will be things like "I'm happier on sunny days" and I'll say "yeah..."
As a side note, people tend to be very unimaginative in the initial stages of data analysis: very often they just take the data they have instead of thinking what data they need to have. And obviously the results correspond to the initial limitations. So much effort is put into data munging and other stages when your inputs are incomplete or plain irrelevant to what your business actually needs.
If you think you should be feeling good today because you're well rested, you probably will feel better.
This year I added in daily mood tracking, and it's mostly focused on taking 5 minutes a day for myself to just jot notes down quickly. I'm using dailo and I have a few categories too (work good/ok/bad - food homecooked/takeout - vices) and just the space I've created to do that has been worth it. I don't expect to get anything too interesting long term besides patterns in takeout/cooking, but I find myself more connected to my day good or bad, and that presence of mind to combat the autopilot is worth it.
The signs I easily ignore are a little bit of brain fog, some reduction in physical energy, and dry skin. The dry skin is meh, and the other two are solved* with food that provides energy.
* = It does not actually resolve it, but it does “enough” and has a long enough lead time that it’s easy to have an instinct of “Oh I’m feeling run down, time to grab a snack” followed by “that didn’t quite get where I expected, I’ll give it time to digest” before forgetting about the whole thing as the project reabsorbs my attention.
There were 3 customizable preset capacities, but even using those required a couple taps each time. Drinking a non-standard amount and manually adjust it required extra effort and taps. Sometimes I’d forget to record it right away, then have to try to remember if I was double-counting a refill. The mental burden was clearly out of balance with the (practically nonexistent) reward.
I quit tracking it when I found myself adjusting my behavior, like avoiding the natural habit to top off my water bottle when it was less than half full and I was getting up from the desk anyway. Ultimately it would be interesting to have that data, but it’s only worth it to me for zero cost/effort.
From there, it's also valuable because it's clear that in the past I might have a dehydration headache, eat something, and still have a headache because I was thirsty. By just incorporating a routine and simple 1 click tracking, that consistency has helped me to easily manage this. But as you said, for you it works "naturally". For me, I was pretty out of tune with that.
Didn't do it for politics, just for lolz and prototyping a concept. Got to Twitch's advertising unlock, but stopped running it because it was a headache to maintain with my home setups/remotely.
But yeah, not trained on my comments, but someone else's, and it didn't take a ton of content to get it going for gpt-2.
Edit: Here's a clip for reference
https://clips.twitch.tv/LovelyObedientTaroNerfBlueBlaster
However, if enough metrics were automatically collected (when did I fall asleep and heart rate lower?) and there was some system that could sort through them (eating a 4th meal increases odds of ruby team scoring additional points by a factor of 1.3x) then you could possibly learn some neat things.
Some of the most interesting things have been pointed out to me by others. Things that are related but I didn't notice. Foods that seem to increase my chances of sickness and things like that.
Conversely I can see when I have been likely to have been drinking from my heart rate.
I think the key here is the passive data collection with intuitive data presentation to drive low barrier to entry analytics.
He has to be in the top 1% of polluters, given his obsessive flying and consumption of technology.
Dozens if not hundreds of charts, depicting every single thing except the thing that is destroying our world.
I filed an issue on his github.
I'm tracking my life by a similar system than OP, but the error with tracking is to start tracking useless things. For example OP track his calorie intake, it is a lot of work, and it is not really usefull at the end. It is unlikely to correlate with any other measures he is doing (apart from weight).
I personally only track mood, activities and position. Each tracking has a specific goal, and only take me 1 minute per day. I have a lot of headaches and could be tempted to track headaches to understand what triggers them. But now that I know what triggers them (when I'm hungry, or I spent too much time exposed to the sun) I would never have been able to detect that correlation from data.
So my advice to people wanting to track their life, identify first what you need the data for, and increase the number of data you track if you have a lack of data to achieve a specific goal, but don't track for the sake of tracking.
For example I track my position, to know where I have been, but also to warn my parents in case I go missing and nobody know where is my last position
I track my mood, to know if I am doing better over time, or if I'm able to improve my sleep not only by feeling it, but by having numbers on
I track my activitites, because I want to know If I work too much, if I do enough sport, or if I spend too much time on my computer in a week
Just reflect on yourself at the end of the day.
Tracking calories is very very simple if you use an app like MyFitnessPal. For most foods you can scan the barcode or you can just enter the calorie data from the label.
Secondly, isn't weight a pretty important measure to track? It can impact a lot of things and therefore if you see you start to have too many days in a week that are above your calorie goal you can catch any gain before noticing it on the scales.
Once you have the basics of collect>summarise>publish metrics, it’s trivial to add just one more.
Do people really need every single aspect of their lives measurable to this degree?
Does anyone (other than your mother) really care?
I, for one, would hate for this to become a thing. It’s like a personal Facebook, but with infinitesimally more extra junk that I don’t want to know.
I feel like keeping a rough journal and manually revisiting would be more helpful than quantifying and correlating. But financial tracking can be useful, to a smaller degree, and tracking mood changes over big life changes can be valuable I guess. Otherwise, does feel vanity
Sometimes, it feels like there's so much going on that it's hard to know if it was the sleep, the diet, the events, or something else. That's where the data might be nice.
Although, I probably could do with some life changes, they need to be manageable.
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
But calling it a vanity project seems overly dismissive. Sometimes you start a project and don't know how it's going to end up. Sometimes it's the journey and not the destination that makes a project worthwhile. I think Felix deserves a lot of credit for writing up his conclusions and sharing them.
it was a vanity project - Felix just found out that it's not worth it later than sooner
Enlightenment
What's your problem with vanity projects? And since when is doing side projects for your CV wrong?
> Do people really need every single aspect of their lives measurable to this degree?
No, but it's fun? "Do people really need X" can be said about anything.
> I, for one, would hate for this to become a thing.
No-one is forcing you to look at it and even less participate in it.
My take on this is Timelinize[0], which is capable of downloading all your data from various digital records like social media, your phone, arbitrary files and unstructured folders, your Google account, etc. So it can aggregate data from Google Photos, Location History, Twitter, Facebook, text messages, and more, onto your computer, organized into a single timeline.
(It is the successor to Timeliner[1].)
I suppose Timelinize can support personal body metrics from various devices, and it is planned to support manual entries as well like journals. Analyses like his would also be possible.
Data can be automatically imported on a regular basis via APIs (like, backing up your new tweets every day). The arbitrary file importer can also be scheduled to watch a directory for changes: just write a file or folder to that directory, and Timelinize will automatically import it (and optionally delete it when finished).
Timelines are stored locally on your own computer, as plain files on disk, with a sqlite database as an index, so it should be a breeze to back up and use/read without proprietary software.
The idea is to:
- Back up your precious data that is otherwise held more-or-less hostage on somebody else's computers
- While creating a detailed, personal journal for your family history
- In a way that can show you your life in context with others' timelines and the wider world.
[0]: https://twitter.com/timelinize (No actual website yet.)
[1]: https://github.com/mholt/timeliner
Hoping it can suck in my missives from Facebook, Twitter, Reddit and Hackernews as a minimum!
Twitter yes, Facebook somewhat (for now -- the API and data graph are very complicated!), Reddit probably not initially (but on my lists; need to learn how to get the data), and Hacker News probably eventually (also need to learn how to get the data).
Why did you stop making updates to Timeliner?
(Also, I'm in the market for a better, available name.)
My own take on this problem is Chronomize [0], which I've been working on since 2020. It's interesting how we independently came up with similar names.
Feel free to reach out if you'd like to chat about personal data management.
[0]: https://chronomize.com
I like your projects' name better too... would still like to see if I can find a better one for mine that is available.
[0] https://tylerfreedman.com/hindsight
1. I've tried out many todo apps and they all aren't quite right for me (atm it's just a pen and paper), so I want to try design UX that's tailor made for me.
2. I found that arbitrary points and day streaks go a long way in making my habits stick. So I want to have an app that can add points/ streaks to anything that I do. (e.g. I complete "go to the gym" from my todo list, and it automatically adds a log to a table. Then that log is parsed to show some arbitrary point & streaks for going to the gym)
3. I want the app to have a REST endpoint that I can use to integrate with other apps to automate data entry.
Honestly, it's more of a procrastination thing than a productivity thing :p but have a lot of fun thinking about/ building the app.
My wife tracks just about everything in her life - and me almost nothing. I’d like the narrow the gap just slightly!
When I was looking at the stats, I realized I can never make something like this. Just alone the list of flying spots mentioned are huge for me. I don't think I have travelled at much internationally. Having a location graph for myself would be mostly few cities and maybe 1 or 2 countries(connecting flights).
Maybe 1 day I get to travel a lot. :)
* Please note that a Happiness score of at least 70% must be maintained to retain employment. Additionally, you must think about Megaco in a positive way for at least 2% of your out-of-work hours."
All bleak jokes aside, interesting work by the author.
Automation is key for 99% of the data with only a small amount requiring manual entry.
I really hope that the world stops idolizing the digital nomad lifestyle.
Either I'm just being jealous, or he is indeed this vain, or both.
edit: it is not, he uses about 3x as much carbon flying (14.2 tons of CO2) as emitted by a typical passenger car per year (4.6 tons of CO2).
> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.
[1] https://www.theguardian.com/environment/datablog/2009/sep/02...
It's a fine balance between caring about stuff and wanting to raise awareness while also seeing other human beings for who they are, assuming good intent.
I disagree (although 'I fly lots' would be a strange thing to ever be 'proud' of imo, regardless of the climate issues). While we should all be mindful of our individual impact, asking people to restrict themselves personally (or shaming them) on such an important and positive freedom as being able to travel widely just sits wrongly with me.
There are much lower hanging fruit as regards climate change to tackle first at a much higher level (e.g. government, multi-national business). I feel our focus should be on that. If you want to make personal sacrifices that's noble but shaming others for not making the same sacrifices is petty and makes it seem like your reasons for making the sacrifices are for appearance rather than because you care. NB: I mean this more generally, it's not wholly directed at your specific comment which wasn't super judgemental.
Maybe, besides political leaders and businesses, that should be limited to inner circle family and friends with whom you have a more complex relationship, and where there is a little bit more receptiveness towards critique. Even in that context it's probably better to just live the lifestyle yourself and hope they will follow.
On the other hand, if you put something out there, to be discussed publicly on HN, I don't feel obliged to only give positive feedback necessarily. Also, the author picked this up in a really nice way and has already added carbon stats to the page.
But man, it's so hard if you deeply care about something to keep your mouth shut. I don't know if you have any topics that make you feel like that?
Edit: actually responding to the rest of your comment, I'm a strong believer in the power of individual choices. The whole plant-based movement for example in The Netherlands (and outside) over the last few years was fully driven by individual consumer choices and little bits of money shifting in other directions. European politics actually tried to hinder it for a while, but at some point when something becomes the popular thing to think, politics will have to follow.
I agree. There are certainly climate-related things I try to hold my extended family to account over (e.g. basics like recycling). I guess where we disagree is where the line is and how much criticism we’re willing to accept and impart :)
Thanks for discussing it with me so pleasantly.
In other words: I am not saying that he should be ashamed of being a pollutant. But I am wondering why he isn’t ashamed of it.
Btw, he isn't traveling just for the sake of traveling — he has conferences around the world, and he's a European working in USA which means he probably flies back and forth from his home country a lot more than an average American. (Doesn't help that North America is far away from everywhere else).
For what it's worth, he makes an attempt to schedule all his conferences such that he wouldn't have to keep flying to/from North America in between conferences.
Felix Krause, keep doing whatever it is that you do and enjoy your life.
Then, they actually let you donate/spend a bit of money towards offsetting your individual carbon footprint, towards whatever initiatives you choose (personally donated towards planting trees).
It's not like it would have as much of a profound impact as stopping the mega-corps would, but it's still something and pretty easy to do.
Disclaimer: not affiliated with them in any way, just thought it's a cool idea.
A little = 140€ / month. Wow. What do people do, redecorate every year?
edit:
11 tons of CO₂ per year
That's 64% less than the Germany average for your household size.
I guess the two of us are pretty ecological ;) But I just realized they didn’t ask about heating (afaik oil based and I have only costs, no usage numbers), so that might make some difference.
Buy one new couch and there goes your budget for the year.
- The data is stored in my own, private database, not connected to any service - Much of the data visualised on that page, is data many larger companies already have about you. I figured, why wouldn't I want to benefit also? - The data shown above seems very personal but doesn’t actually expose any real sensitive information. For example, disclosing your current location, your home address, or stores and places you frequently visits is sensitive and potentially dangerous. Even looking at the map of my movement data on my page, you won’t get any more information that what most people have in their public Twitter bio.
I made sure all graphs are non-interactive, and had all personal information stripped out.