As we were chatting a lot, my phone’s auto-fill started picking up that after ‘Red’ I intended to type ‘Crab’ quite quickly, and the ‘Comedy’ after ‘Divine’. However, it never managed to get the ‘Horse’ after ‘Giant’. Why is that? Was the next-word prediction just not good enough?

In today’s short learning adventure, we’ll talk about a lesser known but crucial effect which protects your privacy in the age of AI when you’re on your phone, tablet or smartwatch — Secure Aggregation.

We’ll get there in 3 quick steps:

1. AI? On my phone? Where? 📱
2. You are just part of the story. 💾
3. Why we perturb. 🔀

Ready? Go!

1. Your phone’s part-time job 📱

What’s your preferred app for catching up with pals and family? Whatever it is, it likely has a text prediction feature to help you type quickly, a spellcheck to help your grammar and, likely, a lot more. If you’re interested in how these work — they’re besides the point today — I wrote a blog on spellchecks in this same series, and the predictive element is a sort of content generation — see a blog on this topic here. But those are other stories!

What it means for us is a quicker, more effective, and even tailored messaging experience. The “tailored” part is important… your phone very much learns stuff about you, when it’s not busy with its main job of helping you scroll Medium and Reddit (speaking from experience). What does it learn?

Typing style (vocabulary, top emojis, punctuation), preferred topics of conversation, languages, … (and more)

… and all of these can be used, as in the pubs example at the start, to learn from and about in real-time, in a single chat. So:

(Machine) Learning is happening on the side (eg: next word prediction)

So what?

Well.. the core of Machine Learning (ML) is that data; your data; is used to train a model. Are you happy with ALL of your data to be trained on? Must you never mention Personably Identifiable Information (PIIs) when chatting with friends? If you ever do, whilst some learning can stay in your phone, analytics may be sent over to a server (and be intercepted, or even used at the other end) to build too accurate of a profile for you, without your knowledge. What can we do?

2. Where is data sent? 💾

Well, not all data is sent. A lot of data stays with you, in your phone, and it’s only processed locally. In particular, when data is sent to a server for analytics, private information undergoes anonymisation processes (de-identification). Again — the how is beyond the scope today :D (maybe a future blog?)

But even so, there are risks of re-identification. Some anonymisation processes are reversible by design. With ML, however, we don’t need to send data directly….

Models generate numerical, uninterpretable quantities through their processes called parameters. These are used to tune the model and they iteratively update themselves. To update a model is, therefore, to update its parameters — which no longer contain any private/sensitive data (matter of fact, nor anything resembling “data” the way it originally was) at all.

There’s a lot more to say here, but the process above describes the logic. (If interested in this topic, read more about Gradient Descent). The gist is:

Your private data never leaves your phone, outside of the form of mathematically sound but nonsensical parameters.

This is a concept known as Federated Learning. Is this great? YES! Is it perfect? Well…

3. What’s Secure Aggregation? 🔀

Whilst difficult, a “Membership Inference Attack” is a form of parameters (gradients) interception where a malicious agent tries reconstructing the original data. In less fancy terms, your data is anonymised and never leaves your phone in raw format; but its parameters may still be used.

Here we come back to the Giant Horse 🐴 pub at the start. Whilst fictional (but believable, right? I can see a pub named that.); what if — instead of a pub — it was an exact location? (street, postcode, everything). It’d get anonymised, only ever used in parameters form, but pose a risk if remembered or intercepted. Not good!

Models can and will happily remember everything. But we can sometimes force them not to. This process is broadly called perturbations, or adding noise to a model. In essence, you’re making it worse by slight modifications of the data, so it cannot exactly pinpoint customers or breach confidentiality by over-profiling.

One form is combining user data: Giant Horse, Giant Tiger, Giant Mammoth.. (ok, last one, less believable) are all pubs — but which one will I go to? Your phone shouldn’t know everything about you, so combining all its knowledge across users like this is a form of intentional perturbation called Secure Aggregation.

Next time your word-predict or autocorrect fails you, take a moment to think if what you’re typing is sensitive in nature. If it is, it might just be in your best interest! 😌

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)

Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro

Show of hands if you’ve used Generative AI, such as ChatGPT, Gemini or DeepSeek? 🙋Good, full class (or almost).

Now, keep your hand up if you know how GenAI works? Oh…😅

If you’ve had to put your hand down, don’t feel bad! It’s a complex topic. Your role may well benefit from using the newest* (I say newest.. we’ll see about that) shiny tech, but “I’m not a data scientist, I don’t need to learn how it does what it does!!”. To that audience I say… where’s your sense of curiousity and wonder? 😛

OK, jokes aside. If you’re interested in a quick (no technical expertise required) primer into how GenAI does what it does, join me over the next few minutes! Disclaimer: If you’re instead looking to upskill your use of these technologies (eg: How do I write better prompts?), or interested in the mathematical details, this is not the right blog for that.

Three steps to knowledge:

1. How does “AI” work to start with
2. What makes “Generative AI” any different
3. So — how does GenAI work then

1. How to predict the future 🔮

In a great simplification: Predictive modelling (of which the infamous Machine Learning is a branch) is most commonly used as ‘supervised’ — You pick something to predict, and you try to do so using other, related things you have access to. The “something” is called a target (as it is the goal of your inquiry), and the things you use are called features. (both have other names, that’s besides the point of today).

Oh — And Machine Learning is just a subset of AI (though the two are often wrongly used interchangeably), and so is Generative AI. Here’s a cool blog explaining this all if you’re interested, credit to the author.

Imagine you’re stuck in a meeting, and you have no windows immediately around to answer this for you: How can you predict the weather outside? Well.. usually, just look it up. Though “local” weather reports are not always exact to your location (if you happen to live in England like me, you know how quickly everything changes). But there are other indicators: Temperature, season, how people are dressed and — importantly — what was the weather like 2h ago, when you last went for a coffee? ☕

All of these (and many others) will give you a good indication of what the weather likely is outside. The current weather is your target, and all the indicators mentioned above your features. It won’t always be spot on, but decent enough! In a nutshell, this is Machine Learning:

The study of updating algorithms, fit for inferring future outcomes based on observed data.

Absolutely not the definition, but it captures what we need. OK — So: AI is used to predict, based on related stuff. How is Generative AI, a subtype of AI, different then?

2. What’s special about GenAI 🎢

Nothing. There are many misconceptions surrounding GenAI, so I want to quickly address two:

• How it works is SO different from other AI!

Nope. Still predicts stuff. OK, let’s be pedantic — when “stuff” is collected and stored, we call it data. It uses data, to predict data. (We’ll get into the how next section)

• Generation of unstructured data (images, text, sound etc.) is so new!

It’s a lot better than it was. But new? Depends on how you quantify that I suppose. Ever heard of Eliza? Or perhaps Google’s old (and really funny sounding) chatbot?

But why is it recently indeed that we’re all amazed by how good it is? Two main reasons, one easier to grasp than the other. The first one: Data. The more stuff (now we know, data) you have, the better you’ll be at predicting related issues. We’ve collected SO MUCH data over the years, it’s astonishing.

The second: Context. Remember the days Google Translate was amusing, if not ridiculous at times? Long gone. If you’re more technically versed, the words you’ve been likely itching to see me write are: Attention mechanism.

Imagine the weather example from earlier. It’s been sunny all day long, the news report you’ve seen this morning showed clear skies, and yet… you step outside, and a rain spell out of nowhere! You should’ve expected it. Afterall, you are in London. Unpredictability is the name of the game here, and besides, you’ve seen those nasty clouds hovering hours ago…. ☔

Why is all of this relevant? Because of context. GenAI uses a mechanism for enforcing the importance of context in its surrounding text/image/whatever generation called attention. In short:

The models will pay respect to the most relevant/important aspects of the features and surrounding context.

This, is indeed new. You can read more about the technical deets here, but for our purposes just remember — context & lots of data.

3. OK — but how does it do it? 🌌

Summary of what we know so far:

• GenAI predicts a “target” using data.
• GenAI uses an “attention mechanism” to enforce context.

So, the only question really is: What is the target, and what is the data it uses to predict? The answer, of course, differs depending on what we generate (images? text? videos?). However, everything on this level of data science can be represented as vectors. Whilst we’re not going to bother with what those even are, understand that — if you know how “text” is generated, you can expand further (since it’s all vectors!).

How is text generated then?

By predicting two things:

I. The missing word in a sentence, based on the other words and overall surrounding context.

> Principle: You purposefully hide/mask a part of a sentence, and have your model predict it. What would make sense there? Have I seen similar things before?

This is known as Masked Language Modelling (MLM)

In the example above, without any more context, “flowers” seems most likely. “cars” is a far cry, but “broccoli”, whilst a bit obscure, is possible. This is a large part of the reason text generation holds some degree of arbitrariness to it — there’s no “correct” choice to this MLM problem.

II. The next sentence, based on the ones before and surrounding context.

> Principle: You feed the model 2 options, and have it pick the one that most “logically” follows a given sentence. What would our normal human texts usually look like?

This is known as Next Sentence Prediction (NSP)

In this instance, there should be a clear winner.

And that’s it!

There is so, so, SO much more to learn about GenAI, but you should now, hopefully, feel more confident the next time you use your favourite LLM that you understand how it does its job.

Hope you enjoyed, see you next time. 👋

This blog was NOT written with the help of any Generative AI. Though it would’ve certainly helped =D

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)

Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro

But how do we actually do that? And more importantly, how do navigation tools like Waze or Google Maps do that, at such high precision?

Today’s short blog: Why graph theory is to thank for making it on time for that important meeting on an early Tuesday. ☕

1. The logic

So, you want to get from your place to the closest coffee shop. Routing complete: About 8 minutes drive, enjoy! 🚗 But… how??

In its simplest form, it’s a game of finding the shortest path between two points, with one added rule: traffic laws.

The most important ones, for our scope, include direction of travel, speed limits, junctions and of course, length of travel. The length is not a straight line though, but rather a combination of multiple, connected streets. This makes for some interesting and unexpected quirks, such as: Turn right now ➡️.. despite your final destination being leftwards of your current position!

Using a combination of the above, the shortest available and valid route is chosen, and the estimated time is calculated as a function of its length (in km) and existing speed limits (in km/h)*.

*Plus A LOT more things.. we’ll soon get to!

But — how do we pick the route to begin with? Here we turn to math, and algorithms like Dijkstra’s to help us choose the shortest path.

2. The math

Let’s first define two crucial concepts:

• Nodes/Vertices: Think of every time you have the ability to take a right or a left, instead of going straight. That’s what we call an intersection or junction.
• Edges: This is a (part of a) street you travel, from one junction to the next one.

In other words, vertices are your decision points (keep going straight? make a left?), and edges represent the distances you travel. Clearly, more vertices add complexity, whilst more edges add distance and, consequently, time.

But we know streets are not all the same length. Therefore, we want a way to penalise longer streets. This final concept is that of a weight. Think of weights as penalties for longer streets, but also for intersections (since they’re often accompanied by traffic lights).

So — What happens in the background is now a strictly mathematics problem:

How can I reach my destination with the smallest sum of weights possible.

In other words, the roadmap system is treated like a graph/network of streets (edges) and intersections (vertices), and our trusty navigation system is solving the aforementioned problem with it. You might also care about visiting the fewest edges (want to only use main streets), or maybe fewest intersections (don’t like waiting at stoplights!).

Pretty simple, right? Or ….

3. The reality

Our system more closely resembles this:

Of course, it had to be a lot more complex in reality. :D The good news however, we now know how to interpret everything driving throws at us: weights !

The real strength of modern navigation lies in the realtime insights. At the end of the day, such apps are recommender systems, calculating a route based on both the graph elements discussed, but also dynamic insights such as: traffic, estimations (from previous days data), road status, human indicators (other drivers) and many, many more.

The accuracy of these systems is a compounding result of all of the above, especially the precise satellite location helping us recalculate egdes’ weights in a matter of seconds.

Recommender systems is something we can dedicate a whole new blog (or more) to, but for now… I’m just grateful I get to use Waze on every commute. I hope you too trust it a bit more, now that you know a little bit about how it works. :) See you next time! 👋

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)

Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro

“Simply” order these countries alphabetically.

Weirdly, “georgia” comes below “India”, and even “Uruguay” in Python, but not in Excel. Clearly, capitalisation matters. At this point, we’d be tempted to dismiss it as “Python is case sensitive, whereas Excel isn’t” — but what about the “####” characters?

A better phrasing then: Characters matter.

Let’s solve this puzzle for good in 3 quick steps!

1. Follow your intuition

“Ordering” means comparing things against each other, and seeing which comes first. You’ve most likely encountered this strange sight:

It probably upsets you, and it should! These appear to be numbers, and sorting them means comparing them as such. But that’s not what is happening here…

To compare / sort / order things, we need a rule for comparison.

Numbers, for example, we expect to be compared mathematically — where 34 > 6. Dates are compared chronologically, where 20/05/2023 > 18/10/2021. So.. how do we compare text against text?

2. Data types

“Sorting” happens with respect to the data type.

In the example above, the “numbers” are not, in fact, treated as numbers — but text rather! More generally: It makes no sense to compare two elements of distinct data types.

Take the number 5 for example. We cannot say that 5 is less, more, nor equal to the date 05/05/2025. Likewise, we can’t compare it to the text “5”.

So: This explains why “34” > “6” and “georgia” > “Uruguay”. But a big mystery remains:

How is text really ordered then? The rule appears to be significantly different than for numbers or dates…

3. Codepoint values

It does look chaotic at first. The answer boils down to… numbers, again!

To compare (and order) two bodies of text, each character gets assigned what we call a codepoint value.

The value is a number — so the task of sorting characters is reduced to the simple and intuitive task of sorting numbers. The numbers are allocated based on a convention called the ASCII encoding. For example, the letter “A” holds a codepoint value of 65, whereas the letter “a” holds 97. We can then say that “a” > “A” ; since it’s equivalent to stating that 97 > 65 as numbers; which is mathematically sound.

OK.. but what about text longer than a single character?

How is the word “georgia” compared to “Uruguay” then? The answer is simple:

To compare text, comprising characters are compared one by one, via their (ASCII) codepoint values.

Since g = 103 and U = 85 ; g > U :: so “georgia” is ‘greater than’ “Uruguay”, as text. In case of a draw on the first letter, we move on to the second and so on.

If all is tied, and one word / text ran out of characters, the shorter word will come first. This is because a non-character is smaller (codepoint) than any character. (see example above)

One final example: What about the “numbers” at the start?

Comparing “34” to “3” as text: their first character is the same. “34” has a second character, whereas “3” doesn’t — so “3” < “34”. When it comes to “6” , “3” < “6” in ASCII so, as text, “34” < “6”, according to our rule.

And that’s all! Thanks for reading, hope you learned something new. ^_^

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)

Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro

I meant heading. *

Readying!! ***

Reading! *****

Hate when that happens. But how does it even work? Let’s learn something new in 3 quick steps:

1. Follow your Intuition

Give credit where credit is due — autocorrect always seems to change words to somewhat similar ones. Let’s stick to our reaping <>reading example. What do we really mean when we say those two words are similar?

They share most of the letters, in the same order.

We can therefore form our first, highly intuitive notion of similarity: Two words are similar insofar as they share as many letters as possible, in the right order. If we want to get closer to quantifying this, we can rephrase it as:

The more similar two words are, the fewer letters we’d need to adjust to get from one to the other.

2. Let’s talk metrics

“Adjust” is a vague word. In our previous example, we’d need to swap a letter for another. Here’s another simple one: Girl <> Girls

In a number of ways, ‘girl’ is no “farther” from ‘girls’ than ‘reading’ is from ‘reaping’. It comes down to what we “value” more. There are broadly two types of adjustments we make to transform words to other words:

• Swaps: Changing a letter to another letter
• Insertions/Deletions: Inserting a whole new letter and/or removing an existing letter.

The question of similarity then becomes:

What is the smallest number of such operations required to transform one word to another?

This is the concept of a Levenshtein distance. The Levenshtein distance between two words is the minimum number of swaps and/or insertions/deletions needed to get from one word to the other. In our examples:

• L(reading, reaping) = 1 ; since we only need 1 letter swap to get there.
• L(girl, girls) = 1 ; as well, since we need a single letter insert/removal.

And that’s it! The smaller the Levenshtein distance is, the more similar the two words are. When two words are identical for example, they have a Levenshtein distance of 0 — which is the smallest possible.

3. Tie it all together

So — how does autocorrect work??

Using numerical notions of similarity like Levenshtein, and comparing against known words in our vocabulary, we can start inferring what someone meant to have typed (more on that at the end). *

Autocorrect integrates this concept, using a strategy known as fuzzy matching. Broadly, fuzzy matching means we have access to a known vocabulary of valid words — so when you mistype (typo), your intended word can be inferred by picking the most similar, valid option from our vocabulary.

Here are two examples:

There’s a lot more to learn about fuzzy matching, autocorrect and natural language — but I hope this gets your interest going!

See you next time :)

*Most autocorrect algorithms also use several other tools & concepts, such as: Context (inferred from surrounding words in your sentence), Human typing (inferred from your letters proximity on your keyboard) and language settings.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)

Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro

Ahh the escape character — my favourite! All you have to do is remember:

The escape character giveth, and the escape character taketh away…

What is it?

A character like any other on your keyboard, but with a special and important mission… Most often — it’s represented by the \ character.

What does it do?

It alters the behaviour of the character immediately after it!

Alters.. how?

OK — story time:

The escape character ( \ ) precedes any other character. If the character it precedes is a metacharacter (or, as I like to say, “has superpowers”!) — it turns it back into a normal character.

Many languages designated metacharacters to perform special commands. For example, * in Word will create bullet points, whilst + in regular expressions is a “1 or more” quantifier. Well, no more! Pair them with the \ and you strip them off their powers — easy peasy! (Examples at the end)

Right! And if it doesn’t have superpowers?

Not so fast! Some of the “normal” characters are really superheroes in disguise! In that case:

The escape character may “activate” the superpowers of a seemingly standard character as well (or as I like to say, remove its disguise!). Not all “normal” characters are latent heroes though, some are just themselves. (and that’s more than enough ❤)

For example, the letter ‘d’ is a seemingly standard, hard-working citizen amongst the characters. Pair it with \ → \d and behold! Many programming languages recognizing regular expressions will quickly tell you that’s a shortcut for “digits” or all characters from 0 to 9.

And that’s all! In short, the escape character:

If it can* : It will swap the metacharacter back to a normal one and vice-versa.

\* Here being that some normal characters have no meta version. :)

Examples

1. Let’s type some reactions on Discord:

Here’s the results!

2. What about “normal” to “special”? Let’s check Python!

This was escape characters in 3 minutes! What should we learn next? Thank you for reading. :)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I’m open to answering any other questions in private — just drop me a message on LinkedIn or Email :)

Find me on: Github || LinkedIn or via email at axl_acc@yahoo.ro