File this under things that should not be this complicated.

I am being rate limited by Docker Hub due to unauthenticated requests. I want to use my docker login / authentication to get a higher limit.

Containerd cannot use the plaintext credentials directly from $HOME/.docker/config.json.

There is conflicting guidance online and how this works across versions 1 and 2 of containerd itself as well as versions 2 and 3 of the toml configuration.

Executive summary:

If you are interacting with a registry that does bearer auth flow (like Docker Hub), you cannot use the 'new' /etc/containerd/certs.d/docker.io/hosts.toml.

This will not work for docker hub:

server = "https://docker.io"

[host."https://registry-1.docker.io"]
  capabilities = ["pull", "resolve"]

  [host."https://registry-1.docker.io".header]
    Authorization = "Basic BASE64_OF_USERNAME_COLON_TOKEN"

Even though there are multiple places that say this is the new way forward. For custom registries that would accept a static header, this probably does work for them.

This warning is listed prominently in the documentation:

NOTE: registry.configs.*.auth is DEPRECATED and will NOT have an equivalent way to store unencrypted secrets in the host configuration files. However, it will not be removed until a suitable secret management alternative is available as a plugin. It remains supported in 1.x releases, including the 1.6 LTS release.

Which is not true when read literally. The new hosts files do have mechanisms to store plaintext credentials and pass them along, just not in the same way that registry.configs did, which is a massive distinction that is not written down anywhere.

The correct way to use existing docker authentication for containerd version 2 / toml version 3 is:

[plugins."io.containerd.cri.v1.images".registry.configs."registry-1.docker.io".auth]
  auth = "BASE64_OF_USERNAME_COLON_TOKEN"

As part of its initContainers, Elastic pulls in os-shell. The question became, how do we specify a different repository for os-shell when we are pulling in the chart externally (and not pulling from disk ourselves). The solution was deceptively simple:

elasticsearch:
  sysctlImage:
    repository: bitnamilegacy/os-shell

What this hides is the rube goldberg machine Bitnami setup to process the above into an image/repository/tag setting you would normally see:

Unpacking the elasticsearch helm chart, we find in
templates/ingest/statefulset.yaml:

      initContainers:
        {{- if and .Values.sysctlImage.enabled .Values.enableDefaultInitContainers }}
        ## Image that performs the sysctl operation to modify Kernel settings (needed sometimes to avoid boot errors)
        - name: sysctl
          image: {{ include "elasticsearch.sysctl.image" . }}
          imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
          command:
            - /bin/bash
            - -ec
            - |
              {{- include "elasticsearch.sysctlIfLess" (dict "key" "vm.max_map_count" "value" "262144") | nindent 14 }}
              {{- include "elasticsearch.sysctlIfLess" (dict "key" "fs.file-max" "value" "65536") | nindent 14 }}
          securityContext:
            privileged: true
            runAsUser: 0
          {{- if .Values.sysctlImage.resources }}
          resources: {{- toYaml .Values.sysctlImage.resources | nindent 12 }}
          {{- else if ne .Values.sysctlImage.resourcesPreset "none" }}
          resources: {{- include "common.resources.preset" (dict "type" .Values.sysctlImage.resourcesPreset) | nindent 12 }}
          {{- end }}

You might think that we then need:

elasticsearch:
  sysctlImage:
    enabled: true
  sysctl:
    image: ...

But no! That is a helper function!

In templates/_helpers.tpl:

{{/*
Return the proper sysctl image name
*/}}
{{- define "elasticsearch.sysctl.image" -}}
{{ include "common.images.image" (dict "imageRoot" .Values.sysctlImage "global" .Values.global) }}
{{- end -}} 

But wait, what is "common.images.image"!?

Another template that comes from the bitnami-common chart:

{{- define "common.images.image" -}}
{{- $registryName := default .imageRoot.registry ((.global).imageRegistry) -}}
{{- $repositoryName := .imageRoot.repository -}}
{{- $separator := ":" -}}
{{- $termination := .imageRoot.tag | toString -}}

{{- if not .imageRoot.tag }}
  {{- if .chart }}
    {{- $termination = .chart.AppVersion | toString -}}
  {{- end -}}
{{- end -}}
{{- if .imageRoot.digest }}
    {{- $separator = "@" -}}
    {{- $termination = .imageRoot.digest | toString -}}
{{- end -}}
{{- if $registryName }}
    {{- printf "%s/%s%s%s" $registryName $repositoryName $separator $termination -}}
{{- else -}}
    {{- printf "%s%s%s"  $repositoryName $separator $termination -}}
{{- end -}}
{{- end -}}

Yaml is the worst.

I solved this once when several months ago when I got new hardware and did not write down the instructions. Kernel did an update, now my drivers do not work. After some head wall bashing for a day, I have solved the problem again. For reference, I have a 5080 RTX.

Here is the most straightforward way to have Nvidia drivers work for Ubuntu 24.04 with secure boot enabled:

# purge any and all existing nvidia installations
sudo apt purge --purge nvidia-*
sudo apt purge --purge libnvidia-*
# Download NVIDIA-Linux-x86_64-570.86.16.run from https://www.nvidia.com/en-us/drivers/details/240524/
# Run this installer
sudo ./NVIDIA-Linux-x86_64-570.86.16.run

• Choose MIT/GPL (non proprietary)
• Yes to continuing the installation on any prompt that requests it.
• Yes sign the module. Use a new key. Write down the location of the .der file it tells you. The path will look like /usr/share/nvidia/nvidia-modsign-crt-HEX.der
• Yes to update/overwrite X server settings.

Before you restart your computer, you will need to load the public key (der file above) into the Machine Owner Key list.

sudo mokutil --import /usr/share/nvidia/nvidia-modsign-crt-HEX.der

• It will then prompt you for a password. Type it in twice.

• Reboot.

• On reboot, the system will ask you if you want to examine new keys, do so, the signature should match what you were told in the installation (fingerprint, etc...).

• You will be prompted for a password to load the module, use the password you made earlier.

Your nvidia linux drivers should now be loaded and working.

Confirm with nvidia-smi.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.16              Driver Version: 570.86.16      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5080        Off |   00000000:01:00.0  On |                  N/A |
|  0%   29C    P8             26W /  360W |     507MiB /  16303MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            3046      G   /usr/lib/xorg/Xorg                      280MiB |
|    0   N/A  N/A            3428      G   /usr/bin/gnome-shell                     47MiB |
|    0   N/A  N/A            4670      G   /opt/google/chrome/chrome                 4MiB |
|    0   N/A  N/A            4738      G   ...ersion=20250511-180108.192000        120MiB |
+-----------------------------------------------------------------------------------------+

After a hardware upgrade, I installed Ubuntu 24 to dual boot with Windows. On startup, I noticed that the terminal would not imediately start. Chrome would immediately start, but network connections would hang. This eventually resolves within a minute or so, but this is completely unacceptable.

Courtesy of this Reddit comment / stackoverflow post:

sudo apt remove xdg-desktop-portal-gnome

was the solution.

Looking at journalctl prior to running the above remove command:

Mar 23 10:12:23 knowles-AMD systemd[2981]: Started xdg-desktop-portal-gtk.service - Portal service (GTK/GNOME implementation).
...
Mar 23 10:13:03 knowles-AMD systemd[2981]: xdg-desktop-portal.service: start operation timed out. Terminating.
Mar 23 10:13:03 knowles-AMD systemd[2981]: xdg-desktop-portal.service: Failed with result 'timeout'.
Mar 23 10:13:03 knowles-AMD systemd[2981]: Failed to start xdg-desktop-portal.service - Portal service.

No side effects or issues thus far. Only remove the one package, do not remove anything that stems from it dependency wise.

from google.colab import drive
drive.mount('/content/drive')

You will be asked to give Colab permission to access your Drive. If you have previously given permission, you still need to click through the entire prompt.

If this is successful, you will see your drive mounted in /content/drive in the file explorer on the left side of the notebook:

You should now be able to browse your files.

NOTE: If you want to share something from ‘Shared with Me’ or something that is not in the root level of your drive, the easiest way to access this is to make a shortcut and put it in your root level drive directory:

It will then show up in the file explorer to the left of the notebook:

It looks like this in your actual Drive UI:

You can then interact with the files with normal OS python code, such as:

import os

directory_path = '/content/drive/MyDrive/some/folder'

def list_files_recursively(f, path):
   for root, dirs, files in os.walk(path):
       for file in files:
           f.write(os.path.join(root, file))

with open('/content/drive/MyDrive/file-listing.txt', 'w') as f:
 list_files_recursively(f, directory_path)

Thankfully, libraries that have become defacto standard lib have well known configuration.

i.e. The requests python library uses

export REQUESTS_CA_BUNDLE=/path/to/bundle

I was running into an issue where an application was using an OpenAPI generated API in python. When attempting to connect to our service using an internally signed certificate, it failed validation.

No worries, I thought, this happens all the time, I just need to find what environment variable OpenAPI needs to add this post code generation / at runtime.

This does not exist.

We are admittedly using an older version (4.3.1) of the openapi-generator-cli. But this configuration option does not exist in the latest version (7.10.0).

For version 5, they did make it better where the ssl_ca_cert will be None and this will force urllib3 to load the system certificates (this would solve my problem as we already update the OS CA bundles in the Dockerfile).

I would prefer to see it default to an environment variable check, falling back to None if it doesn't exist, to then fall back on the OS CA bundle. This would avoid code updates when trust needs to be changed.

Client scope -> click on profile

Mappers tab -> Add Mapper (By configuration):

Click on Group Membership

Fill out the mapper details. The Token Claim Name is what your backend code is looking for in the json (if you do not control this, it needs to match).

Happy integrating!

Upgrading a "legacy" Spring 5 / Java 11 SPA to Spring 6 / Java 17 was a bit of a to do.

Things you may run into with Spring Security:

1. WebSecurityConfigurerAdapter is gone.
2. Ant matchers no longer exist. You have to set a config item to use the previous ant path matching behavior:

spring:
  mvc:
    pathmatch:
      matching-strategy: ant_path_matcher

3. CSRF requires a lot more code to get the default behavior we had before in a single page application.

.csrf(c -> c
.csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())

.csrfTokenRequestHandler(new SpaCsrfTokenRequestHandler()))
.addFilterAfter(new CsrfCookieFilter(), BasicAuthenticationFilter.class)
                
// and CsrfCookieFilter / SpaCsrfTokenRequestHandler classes from the above SPA link

4. @Component no longer registers API endpoints, you need @RestController or another annotation.

This is the first official release from the upstream repository that contains the update to a modern maria!

If you are working with MariaDB in a Java 17+ toolchain, definitely check this release out!

I had a web server being served by one geventwebsocket.gunicorn.workers.GeventWebSocketWorker:

gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 --threads 100 -b 0.0.0.0:5000 'app:create_app()'

This was not keeping up with both the UI and API requests, so I bumped up the worker count:

gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 5 --threads 100 -b 0.0.0.0:5000 'app:create_app()'

This resulted in web socket connections constantly thrashing because there was no guarantee that the initial handshake worker would get subsequent requests.

From https://flask-socketio.readthedocs.io/en/latest/deployment.html :

Due to the limited load balancing algorithm used by gunicorn, it is not possible to use more than one worker process when using this web server. For that reason, all the examples above include the -w 1 option.

The workaround to use multiple worker processes with gunicorn is to launch several single-worker instances and put them behind a more capable load balancer such as nginx.

Gross. Okay.

Script to stand up multiple app servers:

gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 --threads 100 -b 0.0.0.0:5000 'app:create_app() '&
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 --threads 100 -b 0.0.0.0:5001 'app:create_app()' &
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 --threads 100 -b 0.0.0.0:5002 'app:create_app()' &
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 --threads 100 -b 0.0.0.0:5003 'app:create_app()' &
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 --threads 100 -b 0.0.0.0:5004 'app:create_app()'

Nginx has to use sticky sessions when it round robins across these app instances, otherwise websockets will not function correctly. This is achieved by using ip_hash in upstream:

http {

    upstream gunicorns {
        ip_hash;
        server 0.0.0.0:5000;
        server 0.0.0.0:5001;
        server 0.0.0.0:5002;
        server 0.0.0.0:5003;
        server 0.0.0.0:5004;
    }

    server {
        listen 443 ssl default_server;
        root   /usr/share/nginx/html;
        ssl_certificate /path/to/crt;
        ssl_certificate_key /path/to/key;


        location / {
            client_max_body_size 200M;

            auth_basic           "Application Login";
            auth_basic_user_file /etc/nginx/htpasswd;

            proxy_set_header     X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header     Host $host;

            proxy_pass           http://gunicorns;
        }

        location /socket.io {
            proxy_http_version   1.1;
            proxy_set_header     Upgrade $http_upgrade;
            proxy_set_header     Connection "upgrade";
            proxy_pass           http://gunicorns;
        }
    }
}

But wait, there's more!

Flask-SocketIO has to be configured with a message queue:

socketio.init_app(app, cors_allowed_origins="*", message_queue='redis://')

Windows support is now fully enabled and tested.

It also fixes some tmpdir issues for Windows Java 11 users (2.7.x releases) that were fixed in the main upstream branch (Java 17).

I found this library whilst googling around and it is phenomenal. A straightforward library to be able to listen to MariaDB events and perform actions on them. Only issue is that it was only deployed in jitpack. Nothing wrong with that per se, but infrastructure wise it is much more likely your organization has a mirror of maven central.

To that end, I forked this project and put in a couple bug fixes, but most importantly it is now released into maven central:

<dependency>
    <groupId>dev.atchison</groupId>
    <artifactId>mariadb-cdc</artifactId>
    <version>1.0.4.2</version>
</dependency>

Release https://github.com/TheKnowles/mariadb-cdc/releases/tag/1.0.4.2

You may run across a use case where you do not know exactly what data you are receiving and cannot make a POJO for Jackson to model. In this instance you may want to simply convert all non null properties. We are Jackson-lite in this instance. We can also provide a list of fields we know we do not want to convert:

  public Map<String, Object> getNonNullProperties(
      final Object objectToIntrospect, List<String> denyListFields) {
    final Map<String, Object> nonNullProperties = new TreeMap<>();
    try {
      final BeanInfo beanInfo = Introspector.getBeanInfo(objectToIntrospect.getClass());
      for (final PropertyDescriptor descriptor : beanInfo.getPropertyDescriptors()) {
        try {
          final Object propertyValue = descriptor.getReadMethod().invoke(objectToIntrospect);
          if (propertyValue != null && !denyListFields.contains(descriptor.getName())) {
            nonNullProperties.put(descriptor.getName(), propertyValue);
          }
        } catch (final IllegalArgumentException
            | IllegalAccessException
            | InvocationTargetException e) {
          // handle
        }
      }
    } catch (final IntrospectionException e) {
      // handle
    }
    return nonNullProperties;
  }

This is a good stopgap until a more formal ICD is formed or data collapses down into a known entity.

Initial inspiration from Stack Overflow.

I am pleased to announce MariaDB4j 2.7.1 ships with this new functionality.

DBConfiguration.java has been updated with a setter for the bin log and everything is handled behind the scenes. You can have the binary log write events with as little code as

  @Bean
  public MariaDB4jSpringService mariaDB4jSpringService() {
    MariaDB4jSpringService mariaDB4jSpringService = new MariaDB4jSpringService();
    DBConfigurationBuilder dbConfigurationBuilder = mariaDB4jSpringService.getConfiguration();
    dbConfigurationBuilder.setBinLogEnabled(true);
    return mariaDB4jSpringService;
  }

in your application.

I opened a PR against the original project here: https://github.com/MariaDB4j/MariaDB4j/pull/710

• MariaDB changed their download urls since the 10.3 commit to [MariaDB4j repo].
• MariaDB 10.4 removed no password root user.
• MariaDB 10.5.2 onward swapped how they did sym links between mysql* and maria* equivalents.

Unfortunately, the main branch has already progressed to spring boot 3 / Java 17 and no maintainence branches will be made. To that end, I have created an effective 2.7.x release branch to support Java 11 / Spring Boot 2.7.x / Spring 5 for the wide swath of existing software that has not or cannot jump to Java 17 / Spring Boot 3.

This is a mostly hard fork of MariaDB4j to account for disconnects between Java 11/17 and Spring 5/6.
The last binary update that shipped with MariaDB4j out of the box was 10.2.11, released in 2017.

Using Spring 5/equivalent JPA there are SQL syntax problems that make version 2.6.0 unusable for modern development.
The only current way to get this to work is to use installed binaries on the host system.
This fork aims to create and maintain a 2.7.x release series to account for the above aforementioned issues.

Linux support is tested. Windows support is simulated. OSX support is not provided.
Best effort will be made to maintain upstream updates as applicable.

Compatability Chart

MariaDB4j	Spring	MariaDB
3.0.0	6	10.2.11 / Installed version
2.6.0	5	10.2.11 / Installed version
2.7.x	5	10.6.12+

Release v2.7.0 · TheKnowles/MariaDB4j

2.7.0 This is the initial release in the 2.7.x maintenance branch. It aims to serve a stopgap for Java 11 projects that need a more modern version of mariadb while using the embedded functionality...

TheKnowlesGitHub

<dependency>
  <groupId>dev.atchison.mariaDB4j</groupId>
  <artifactId>mariaDB4j</artifactId>
  <version>2.7.0</version>
</dependency>