← Home About Archive Photos Replies Also on Micro.blog
  • As a follow up to my last post on Charm’s Logger package, I took a look at memory usage for a predefined logLevel, a raw int 100, and math.MaxInt32. My hypothesis was that math.MaxInt32 is more efficient, but using math.MaxInt32, there were no memory allocations shown by the profiler. Amazing!

    → 9:12 PM, Jan 27
  • How Does Charm Logger Format Logs?

    I woke up today wondering how charm’s logger library (https://github.com/charmbracelet/log) actually handles log levels. I know they have some levels predefined in an enum, but what’re they doing with that? This is relevant to #golang #opensource #programming #readingcode

    logger.Log is the natural entrypoint, as it’s the actual code doing the logging https://github.com/charmbracelet/log/blob/1e6353e3ca793f1177148e09f990ef220e19b037/logger.go#L56

    It dispatches to logger.handle, which has this curious setup:

    func (l *Logger) handle(level Level, ts time.Time, frames []runtime.Frame, msg interface{}, keyvals ...interface{}) {
    
    	var kvs []interface{}
    	if l.reportTimestamp && !ts.IsZero() {
    		kvs = append(kvs, TimestampKey, ts)
    	}
    
    	if level != noLevel {
    		kvs = append(kvs, LevelKey, level)
    	}
    

    I think this means that no matter your log level, whether the log will print or not, the logger always constructs a log line for you, unless you match the noLevel?

    As a note, noLevel is private, but is defined as math.MaxInt32. This probably means the most efficient way to generate no logs is to declare your own equivalent noLog level as a default, but I can come back to that in a few paragraphs.

    So how do the logs get written? Scanning to the end of the function, it’s this straightforwards l.b.WriteTo(l.w). l is a pointer to the logger struct, b is a bytes.buffer inside that logger, and it’s written out to whatever destination is stored in an io.writer called w.

    So if we can understand how b get populated, we’ve cracked how these logs are written. The answer is in this switch statement, though it’s indirect:

    	switch l.formatter {
    	case LogfmtFormatter:
    		l.logfmtFormatter(kvs...)
    	case JSONFormatter:
    		l.jsonFormatter(kvs...)
    	default:
    		l.textFormatter(kvs...)
    

    Each of these formatters accepts responsibility for populating the buffer with an appropriately formatted log output. The logfmtFormatter, for example, begins with e := logfmt.NewEncoder(&l.b), accepting a reference to the buffer.

    So, that’s the story. Logs are appended to a key value store, which is handed to a formatter that writes formatted output to a buffer on the logger, which is then written.

    I’ll come back to the question of figuring out if the best performing no-opt logger is setting hte logLevel to math.MaxInt32 in my next post about this.

    → 7:23 AM, Jan 24
  • Time to update my logo. find . -iname "*.html" -exec sed -i '' 's/logo.png/dolphint.png/' {} \; I never did fall in love with xargs.

    → 8:48 AM, Jan 19
  • The joy of maintaining a website as flat files is mostly found in using sed for sitewide refactoring.

    → 8:44 AM, Jan 19
  • two goals today

    My first goal today is to look at PlayTechnique’s website, simplify the front page to a clearer “this could be a business you trust” message, and figure out what layout works for the combination of blog posts and project docs that’re on there.

    My second goal is to begin the Jinx rewrite I’ve been avoiding for a year. Jinx is such a great workflow for dockerised Jenkins, it’s a shame not to finish it up and get it released. I want it to meet my original goal of rails-like cli for Jenkins containers.

    → 8:17 AM, Jan 19
  • The scary thing in my heart is holding simultaneously knowledge that I am a productive, ordinary person with minor ambitions, and that the legislators of the country are okay legally redefining me as a terrifying monster.

    → 8:11 AM, Jan 19
  • I was sad to find out that Hey email doesn’t support being an SMTP relay. The UI’s really quite nice, but I have a few alerts I need to forward, so it’s on to the next big thing.

    → 11:14 AM, Jan 18
  • Every day I remember my Productive Days and think “I should get an early start on work, because Productive feels Good” and every morning I sit here reading a little book or codebase unrelated to work and feel joy instead of the productive good.

    → 7:49 AM, Jan 14
  • Damn I wish I’d written this - github.com/esnet/gdg… It’s a CLI tool for grabbing grafana dashboards from your instance. I want to take a day off to read it and learn. It uses mockery, Taskfiles, a VHS tape (I haven’t used VHS since rails!), go-releaser. A joyful tool romp!

    → 7:44 AM, Jan 14
  • I started watching videos on prompt engineering yesterday. So far I’m thirty minutes in and it feels like a scam. I keep thinking from reading Simon Williamson’s blog that there’s utility to understanding the models when crafting prompts, but gosh darn it more feels like a prose writing course.

    → 7:07 AM, Jan 13
  • It’s a small thing but this evening I understand the permission in a browser extension’s manifest a bit better than this morning, and the errors caused by incorrect permissions, and that’s progress and that’s enough.

    → 10:46 PM, Jan 9
  • Yay, I said, now we have a log ingester and this is great! But one who had been here before said no, now our troubles begin for now we have to learn yet another unique query language.

    → 7:15 AM, Jan 9
  • I found a neat CLI tool today. Check out man comm

    comm – select or reject lines common to two files

    Try it out! ls > 1.txt && ls | sed '$d' > 2.txt && comm 1.txt 2.txt

    → 6:43 PM, Jan 7
  • What an interesting way to discover I can’t write an html tag directly in micro.blog. </rss> was the missing tag, for all nobody who wondered.

    → 6:39 PM, Jan 7
  • Fixed the rss feed in Andrew. It turned out I was missing a . How simple a fix!

    → 6:28 PM, Jan 7
  • I’m not sure if micro.blog is replacing my quotes with smart quotes, or if it’s something else in the stack. Truly it was irritating when it came to copy/paste my code this morning.

    → 5:17 PM, Jan 7
  • Two projects today: fix the rss feed for Andrew, and put together yesterday’s #fluentbit investigations as a wee go program. First, I should decide if my headache is mellow enough to go back to work.

    → 8:29 AM, Jan 7
  • Moving on from source codem, here’s the log paths reconstructed: ; k get pods –all-namespaces -o json | jq -r ‘.items[] | .metadata.name + “” + .metadata.namespace + “” + (.spec.containers[].name) + “-” + (.status.containerStatuses[].containerID | split("://")[1]) + “.log”’

    → 1:49 PM, Jan 6
  • Ah, the legacy pattern doesn’t contain the namespace. I wonder if that’s why it’s legacy…

    The legacy tests do include the namespace. Odd github.com/kubernete…

    → 1:43 PM, Jan 6
  • I as close! This seems like a good enough reference for the log path github.com/kubernete…

    logPath := fmt.Sprintf("%s_%s-%s", podFullName, containerName, containerID)

    → 1:32 PM, Jan 6
  • When fluentbit retrieves logs, it’s worth knowing two things:

    1. It gets container logs, not pod logs
    2. dockerd arranges for a container’s name in the log file to be podName_namespace_imageName-dockerdContainerID

    I assume different container daemons might give ‘em different names. I’ll check…

    → 1:21 PM, Jan 6
  • The answer was to ask ChatGPT to solve the question for me:

    ; kubectl get pods –all-namespaces -o jsonpath="{range .items[]}{.spec.containers[].name}{'\n'}{end}"

    → 1:12 PM, Jan 6
  • A few minutes later I’ve edited the configmap for fluentbit and restarted the pod. The fluentbit logs now show this:

    [error] [input:tail:tail.0] read error, check permissions: /var/log/justnocontainers/*.log

    Okay, so what’s the quickest way to see all the actual containers using kubectl…

    → 1:08 PM, Jan 6
  • Looks like fluent can already see grafana:

    ; k logs -n fluentbit fluentbit-fluent-bit-km64g | grep grafana >/dev/null && echo $? 0

    Let’s break the config. First, where’re the logs in minikube?

    ; minikube ssh ; find / -iname “grafana” 2>/dev/null /var/log/containers/my-grafana-7fcb66b7d6-tnx4d_monitoring_grafana-dc9be7f97e3f8cb53bce03682d3daf74d9e35bcc1c0d1b0aa8b24238dd7d77a7.log

    If I reconfigure fluent to not look in /var/log, I should be correctly broken!

    → 12:37 PM, Jan 6
  • step 2: install some stuff into k8s ; helm repo add grafana grafana.github.io/helm-char… && helm repo add fluent fluent.github.io/helm-char…

    etc

    → 12:32 PM, Jan 6
  • step 1: get kubernetes locally ; brew install minikube && minikube start

    → 12:32 PM, Jan 6
  • Step 0: does #fluentbit already support this? The docs show the kube apiserver provides pod names, but container names are provided by the file system (https://docs.fluentbit.io/manual/installation/kubernetes). So only if fluentbit is already configured correctly.

    → 12:03 PM, Jan 6
  • I’ve found that it’s too easy to miss in k8s that fluentbit is missing various pods' logs.

    Today’s project: what’s the smallest version of a fluentbit verifier I can build in under a day?

    → 9:44 AM, Jan 6
  • It’s easy to waffle, hard to be succinct. It’s easier to draw attention to a problem, harder to work on a fix.

    → 10:47 AM, Jan 5
  • Today’s progress was to dust off an old Chrome extension and to break and fix it to re-familiarise myself with how these things work. I never feel like I’ve done enough, but I did something and I’ll be okay with that.

    → 5:05 PM, Jan 4
  • Small Disciplines I Dislike Build Things I Love

    I worked with John Arundel for a few months at the beginning of last year, the dude who runs Bitfield Consulting.

    I told him my ambition: to run my own little software shop. He told me I’d do well to start by running a consultancy and using that to get familiar with people’s problems.

    To this day, I find the advice hard to swallow. I want to write my own little programs and be happy. I think he’s right, though, that I’d be better off selling my time for myself instead of selling it for a salary.

    Cal Newport wrote in Deep Work that people who can concentrate are becoming a lost commodity, people who can read and read until they understand are becoming valuable. I don’t know if that’s true, but I do know I spent a lot of last year just regaining my ability to focus. I was sober for about 9 1/2 months, I made myself spend more time reading, I got medications for some focus issues.

    I dream; I dream and dream and get frustrated when it hasn’t helped. It’s time to take advice from Kai Greene instead. He said in a great documentary that a world class body builder isn’t the most knowledgeable person in the room; they’re the person who built out small habits, one day at a time.

    Here’s a beautiful quote from him:

    Those star moments that fill the highlight reels and leave the audience in awe to watch, those seconds are built on thousands and thousands and thousands of hours…basic fundamentals being applied over and over and over again. Getting up a certain time, doing certain things, cooking your meals, keeping the disciplines, keeping a checklist. Those are the things that string together to make a day of efficient action.

    The more of those days of efficient action you string together, the more likely you are to succeed.

    → 12:12 PM, Jan 4
  • I have no love for the man, but god damn if Trump didn’t show me how powerful saying a single message loudly thousands of times is.

    → 10:53 AM, Jan 4
  • January’s Adventure: browser plugins and a running schedule.

    I finished a couple of tutorials for writing chrome plugins, then found ChatGPT and Claude are bad at writing new plugins. Tonight is for learning how to connect to Claude’s API from a plugin.

    Today’s run: 3 miles.

    → 7:20 AM, Jan 2
  • 2025: new year, same goals, new fed-up-with-my-nonsense me.

    Last year in January I wanted to blog more. I wanted to do it in html, css and javascript, but without the tediousness of maintaining index.html lists of all the pages I’d written. I wrote a webserver to handle lots of boilerplate andrew, wrote posts on playtechnique.io and I was satisfied.

    The truth is, though, PlayTechnique looks like a personal blog, not a business:

    Screenshot of the front page of playtechnique.io, which looks like a single column with a list of blog post titles. Needs more zing.

    I started PlayTechnique, my LLC. I failed, though, in getting revenue through PlayTechnique.

    2025 goals

    Finally finish Alan Moore’s book Jerusalem

    I’ve been about 670,000 words in since March-ish and really want to get that last 330,000 words read. The second arc, though, was a real distraction from what I loved in the first.

    PlayTechnique makes money

    I need to stop being scared of the best advice I received: to be self-employed, start as a consultant.

    Commercial Software, Gwendolyn!

    As above, but instead of consulting I want to release things people buy. That sounds lovely.

    javascript’s a reality: it’s time to learn it

    I’ve always been an old unix stick in the mud, enjoying languages that traditionally build CLIs, but gosh I need to get past it and into javascript and frameworks.

    I finally learn to touch type

    I can touch type, but not properly. I use weird fingering, with a bias towards a few fingers. I bought a keyboard.io split keyboard to fix this with.

    I put on the beef

    I’m decently strong for a natural, self-taught lifter, but I’m not lean. I want muscularity to be my thing. My blocker here’s primarily not knowing how to transition from Diet to daily maintenance without immediately putting on fat.

    → 3:26 PM, Jan 1
  • RSS
  • JSON Feed
  • Micro.blog