Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

December 05 2016


Government U-turn on Health Privacy

Now that everyone’s distracted with the supreme court case on Brexit, you can expect the government to sneak out something it’s ashamed of. Health secretary Jeremy Hunt has decided to ignore the wishes of over a million people who opted out of having their hospital records given to third parties such as drug companies, and the ICO has decided to pretend that the anonymisation mechanisms he says he’ll use instead are sufficient. One gently smoking gun is the fifth bullet in a new webpage here, where the Department of Health claims that when it says the data are anonymous, your wishes will be ignored. The news has been broken in an article in the Health Services Journal (it’s behind a paywall, as a splendid example of transparency) with the Wellcome Trust praising the ICO’s decision not to take action against the Department. We are assured that “the data is seen as crucial for vital research projects”. The exchange of letters with privacy campaigners that led up to this decision can be found here, here, here, here, here, here, and here.

An early portent of this u-turn was reported here in 2014 when officials reckoned that the only way they could still do administrative tasks such as calculating doctors’ bonuses was to just pretend that the data are anonymous even though they know it isn’t really. Then, after the care.data scandal showed that a billion records had been sold to over a thousand purchasers, we reported here how HES data had also been sold and how the minister seemed to have misled parliament about this.

I will be talking about ethics of all this on Thursday. Even if ministers claim that stolen medical records are OK to use, researchers must not act as if this is true; if patients end up trusting doctors as little as we trust politicians, then medical research will be in serious trouble. There is a video of a previous version of this talk here.

Meanwhile, if you’re annoyed that Jeremy Hunt proposes to ignore not just your privacy rights but your express wishes, you can send him a notice under Section 10 of the Data Protection Act forbidding him from disclosing your data. The Department has complied with such notices in the past, albeit with bad grace as they have no automated way to do it. If thousands of people serve such notices, they may finally have to stand up to the drug company lobbyists and write the missing software. For more, see here.

October 31 2016



Last week I gave a keynote talk at CCS about DigiTally, a project we’ve been working on to extend mobile payments to areas where the network is intermittent, congested or non-existent.

The Bill and Melinda Gates Foundation called for ways to increase the use of mobile payments, which have been transformative in many less developed countries. We did some research and found that network availability and cost were the two main problems. So how could we do phone payments where there’s no network, with a marginal cost of zero? If people had smartphones you could use some combination of NFC, bluetooth and local wifi, but most of the rural poor in Africa and Asia use simple phones without any extra communications modalities, other than those which the users themselves can provide. So how could you enable people to do phone payments by simple user actions? We were inspired by the prepayment electricity meters I helped develop some twenty years ago; meters conforming to this spec are now used in over 100 countries.

We got a small grant from the Gates Foundation to do a prototype and field trial. We designed a system, Digitally, where Alice can pay Bob by exchanging eight-digit MACs that are generated, and verified, by the SIM cards in their phones. For rapid prototyping we used overlay SIMs (which are already being used in a different phone payment system in Africa). The cryptography is described in a paper we gave at the Security Protocols Workshop this spring.

Last month we took the prototype to Strathmore University in Nairobi to do a field trial involving usability studies in their bookshop, coffee shop and cafeteria. The results were very encouraging and I described them in my talk at CCS (slides). There will be a paper on this study in due course. We’re now looking for partners to do deployment at scale, whether in phone payments or in other apps that need to support value transfer in delay-tolerant networks.

There has been press coverage in the New Scientist, Engadget and Impress (original Japanese version).

October 04 2016


Security Economics MOOC

In two weeks’ time we’re starting an open course in security economics. I’m teaching this together with Rainer Boehme, Tyler Moore, Michel van Eeten, Carlos Ganan, Sophie van der Zee and David Modic.

Over the past fifteen years, we’ve come to realise that many information security failures arise from poor incentives. If Alice guards a system while Bob pays the cost of failure, things can be expected to go wrong. Security economics is now an important research topic: you can’t design secure systems involving multiple principals if you can’t get the incentives right. And it goes way beyond computer science. Without understanding how incentives play out, you can’t expect to make decent policy on cybercrime, on consumer protection or indeed on protecting critical national infrastructure

We first did the course last year as a paid-for course with EdX. Our agreement with them was that they’d charge for it the first time, to recoup the production costs, and thereafter it would be free.

So here it is as a free course. Spread the word!

October 01 2016


"Why do you work in security instead of something more lasting ?"

This post grew out of a friend on Facebook asking (I paraphrase) "why do you spend your time on security instead of using your brainpower for something more lasting ?". I tried to answer, and ended up writing a very long reply. Another friend then encouraged me to re-post my reply to a wider audience. The below is a slightly edited and expanded version. It is much less polished than my usual blog posts, more personal, and somewhat stream-of-conscious-y. Apologies for that.

Why do I work in security instead of on something more lasting?

Predictions about what is "lasting" are very difficult to make :-). I think outside of the exploit-of-the-day, there's lasting work to be done in understanding of exploitation (because machines and automata aren't going away, and neither are programming mistakes), and I sincerely hope I'll have opportunity to do that work.

I tried my hand in cryptography / academia, and found it more prone to political trends/fads and less blindly results-oriented than security - to my great disappointment. When all attacks are of theoretical complexity 2^96, verifying and replicating results becomes difficult, and objective truth suffers (see below).

In the following, I will state a few things that I really like about the computer security community. I did not realize this immediately - instead, I learnt this over many years and engagement in other communities.
  1. Original thinkers. I used to joke that there are less than 2 dozen reasons why security as a field doesn't suck, and I know many of them personally. Now, the 2 dozen is bullshit, but what is true that in all the noise & hype, I have met a number of very fun, unconventional, and deeply insightful thinkers of very different backgrounds. They are few and far between, but I wouldn't have met them without security, and I am grateful for having met them. Many exploits require considerable inventiveness, and non-obvious / creative ways of solving problems; they are sometimes like a good joke / magic trick: With an unexpected twist that makes you laugh in disbelief.
  2. Tolerance of non-conformism and diverse educational backgrounds. There are few other industries where people who did not finish high school mix with people with postgraduate degrees, and debate on even terms. With all it's problems and biases, the part of the community I grew up with did not care about gender, skin color, or parental income - everybody was green writing on a black screen.
  3. Intellectual honesty. When discussing attacks, there is "objective truth" - you can establish whether an attack works or does not work, and checking reproducibility is easy. This is not true in many other disciplines, and "truth" becomes a matter of social consensus - even in pure math, where proof should be absolute. Having objective truth is extremely helpful to prevent a discipline to devolve into scholasticism.
Many other fields which may be more "lasting" do not have the luxury of these three points. Also be aware that my visibility into the security community is very skewed:

My skewed view of the security community

It is common to hear negative things about the community - that it is elitist, full of posturing, or of people that are mean / demeaning to others with less experience. This is not the community I experience - and this discrepancy has been puzzling me for a while.

For one thing, everybody is always nice to me. I am not sure why this is the case, but the only non-niceties I encountered in this industry were in leaked email spools. This makes it difficult for me to notice people being mean to newcomers and elitist - and it saddens me to hear that people are being shit to each other.

People weren't always nice to me - like any group of teenagers, 1990's IRC was very often not a friendly place, and #cracking would kickban you for asking a question. I found a home of sorts in a channel called #cracking4newbies - a very welcoming environment dedicated to joint learning. It was great for me: I could ask questions, and either got answers or links to documentation. A few members of #cracking were no longer active, and held status in the channel for historical reasons, #cracking4newbies on the other hand was full of eager & active youngsters.
I somehow managed to avoid being around the posturing and status games much, and in some bizarre stroke of luck, have managed to do so up to this day. The people in the security community I spend time with are genuinely interested in the technical challenges, genuinely curious, and usually do not care about the posturing part. The posturing may happen at industry conferences, but I tend to not notice - the technically interesting talks tend to adhere to substance-over-style, and the rest is as relevant to me as big advertisements for broken content inspection appliances.

All I want to say with this section is: I do not know how I managed to avoid experiencing the bad sides of the security community much. Some of it was luck, some of it was instinct. There are plenty of things I find annoying about the security community (but that is for another post :-), but in my day-to-day life, I don't experience much of it. If you are in security, and feel that the community is elitist or demeaning to people learning, I hope you succeed in seeking out the (many) people I encountered that were happy to share, explain, and just jointly nerd out on something. Feel free to reach out any time.

On building vs. breaking

I quite often hear the phrase "I quit security and I am much happier building instead of breaking things". This is a normal sentiment - but for me, security was never about "just" breaking things. Tooling was always inadeqate, workflows horribly labour-intensive, and problems were always tackled on the lowest level of abstraction, missing the forest for the trees.
In my reverse engineering classes, I always encourage people to be tool builders. Most of security work today is akin to digging trenches with chopsticks. Invest in designing and building shovels. Perhaps we will even get a bulldozer in my lifetime. Slowly but surely, the industry is changing in that direction: Microsoft is commercializing SAGE, no code auditor is more productive (even though more in-depth) than a farm of computers running AFL - but the discrepancy between the quality and quantity of tools that developers have available vs. the tools that security review has available is still vast.
I like my work most when I can cycle through building / breaking phases: Try to break something, notice how insanely badly the tooling is, cycle through an iteration of tool development, return to the breaking etc.
I realize this isn't the path for everybody, but I don't think that security is "always just about breaking". The most persistent person gets bored of chopstick-trench-digging. Invest in tooling. Being a better developer makes you a better hacker. And perhaps you like building more than breaking, and I can't fault you for that.

My friend Sören happens to be one of the best C++ developers I know. When we first met in undergraduate math class, I described what I do for a living to him (reading code for subtle mistakes), and he said "that sounds like one of the worst imaginable jobs ever". He is a builder, and I have nothing but admiration and respect for him - and from the builder's perspective, his assessment is right.

I still like finding subtle bugs. To paraphrase another person who I respect a lot: "People still search for new stuff in Shakespeare hundreds of years later".

Using security as an excuse for broad learning

I once read that "cryptography gathers many very different areas of mathematics like a focal lens". The same is very true of security and computer science. Security happens at the boundaries between layers, and I have used working in security as an excuse to learn about as many layers as possible: Low-level assembly, high-level stuff on formal verification, and even electrical engineering problems and their implications on security. People talk about "full stack engineers" a lot; security allows me to roam the full stack of abstractions in computer science without guilt. All layers are relevant for security, all layers are interesting in their own right, and each layer has it's own funny quirks.


Given the length of this blog post, it is evident that I have asked myself the question "why do I do this" many times. And I have thought about devoting attention to other things often enough. Who knows, I am 35, so I have about 30 years of professional activity ahead of me - which may be enough to fail in one or two other fields before returning to give grandfather-security keynotes. :-)

But right now, I am actually enjoying having my hands dirty and thinking about heap layout for the first time in years.

September 15 2016


Hacking the iPhone PIN retry counter

At our security group meeting on the 19th August, Sergei Skorobogatov demonstrated a NAND backup attack on an iPhone 5c. I typed in six wrong PINs and it locked; he removed the flash chip (which he’d desoldered and led out to a socket); he erased and restored the changed pages; he put it back in the phone; and I was able to enter a further six wrong PINs.

Sergei has today released a paper describing the attack.

During the recent fight between the FBI and Apple, FBI Director Jim Comey said this kind of attack wouldn’t work.

September 03 2016


Essays about management in large(r) organisations (1): Process and flexibility

Even though I often profess that my primary interests are technical, by this point in my life I have been exposed to a variety of different organisations and management styles: From the self-organizing chaos of the 1996-2002 cracking/hacking groups, through the small engineering-centric startup zynamics, via the various organisations (both governmental and industry) I consulted for at some point, to the large (but nonetheless engineering-centric) culture at Google.

I enjoy thinking about organisations - their structure, how information flows, their strengths and dysfunctions. Part of it may be the influence of my father (who wrote extensively on matrix organisations, but also on organisations that fail); the other part is certainly the recognition that both company culture and organisational culture matter . In any organisation, setting the culture and organisational structure - and keeping it healthy - is paramount, and probably the key element that will allow long-term success. Ignore culture and organisation structure (both explicit and implicit) at your peril.
I had a lot of time to think in the last year, so in the coming months I will write a few posts / essays about company culture and management.
The first post is about organisational processes - why they are important, but also how they can take on a life of their own and strangle flexibility.

A technical anecdote to start with

In early 2004, the first prototype of BinDiff started to work properly - just when Microsoft released MS04-001: A series of amusing little memory corruptions inside the H.323 parsing component of Microsoft ISA server (a now-discontinued firewall product). Using BinDiff on the patch, it was evident that the problems were inside the ASN.1 PER parsing routines in a central library - but instead of fixing the library, the patch fixed the issue inside ISA server. The patch fixed only one exploit path, but the actual vulnerability was still there. This meant that any other program using the same library remained vulnerable, and the patch had now effectively disclosed the security issue. I started searching for other applications that used this library. The first program I found which was also affected by this vulnerability was Netmeeting - Microsoft had inadvertently given a remote code execution bug in Netmeeting to everybody. It wasn't until MS04-011, at some point in April, that this vulnerability got fixed in the correct place -- the library.
The technical details of the bug are not terribly interesting - what is interesting is what the mistake revealed about flaws in Microsoft’s organisational structure, and how they reacted to the bug report.

What could we deduce/learn/extrapolate from this event?

  • Bug reports were likely routed to the product teams - e.g. if a bug is reported in your product, the bug report is routed to you.
  • Responsibility for fixing a bug appears to lie with the product teams (see above), and teams are incentivized (either directly or indirectly through feature deadlines etc.) to get bug reports “off their desk” quickly.
  • Patching shared central code is harder than patching code you own (for various reasons - perhaps compatibility concerns, other priorities from other teams, or perhaps even a heavyweight process to ask for changes in critical code).

What likely happened is that the ISA team decided that dealing with the issue on their side is enough - either because they did not realize that the same issue will affect others, or because dealing with the other team / the library is a pain, or for some other unknown reason. Microsoft’s bug fixing process incentivized “shallow” fixes, so for attackers, finding the ultimate root cause of a vulnerability could expose other vulnerable programs.
This is a classical example of making a locally convenient decision that adversely affects the larger organisation.
From what I heard, Microsoft learned from this event and made organisational changes to prevent similar mistakes in the future. They introduced a process where all patches are reviewed centrally before they go out to ensure that they don't inadvertently fix a bug in the wrong spot, or disclose a vulnerability elsewhere.

Processes as organisational learning

In what an MBA would call ‘organisational learning’, a process was created out of the experience with a previous failure in order to prevent the mistake from happening again. A process is somewhat similar to organisational scar tissue - the organisation hurt itself, and to prevent such injury in the future, the process is established.
Surprisingly, most organisations establish processes without documenting explicitly what sort of failure and what sort of incident caused the process to be established. This knowledge usually only lives in the heads of individuals that were there, or in the folklore of those that talked to those that were there. After a half a decade or so, nobody remembers the original incident - although the process will be alive and kicking .
A process can prevent an organisation from doing something stupid repeatedly - but all too often, the process takes on a life of its own: People start applying the process blindly, and in the hands of an overly-literally-minded person, the process becomes an obstacle to productivity or efficiency. The person in charge of applying and enforcing the process may themselves not know why it is there - just that it is "the process", and that bad things can happen when one doesn't follow it.
My grandfather used to say (I will paraphrase) : "a job with responsibility is a job where you don’t simply apply the rules, but need to make judgements about how and where to make exceptions". This quote carries an important truth:
People at all places in an organisation need to be ...
  1. Empowered to make exceptions: After demonstrating sound judgement, people need to feel empowered to make exceptions when the letter of a process gets in the way of the greater good and changing the process would be excessive (for example, in a one-off situation).
  1. Empowered to challenge processes: The reasoning behind a process must to be accessible to organisation members, and there needs to be a (relatively pain-free) method to propose changing the process. Since powerlessness is one of the main drivers of occupational burnout, this will help keep individuals and the organisational structure healthy.

Some organisations get the “exception” part right - most big organisations only function because people are regularly willing to bend / twist / ignore processes. Very, very few organisations get the “challenge” part right-- making sure that every employee knows and understands that processes are in the service of the company, and that improvements to processes are welcome .
I think that the failure to achieve the challenge-process frequently arises due  to "lack of institutional memory". When organisations fail to keep track of why a process was created, all sorts of harmful side-effects arise:
  1. Nobody can meaningfully judge the spirit of the process - what was it designed to prevent?
  2. Making an exception to the process is riskier - if you do not know what it was designed to prevent, how can you know that in this particular case that risk does not apply?
  3. Amending the process becomes riskier. (Same reason as above.)
  4. Challenging the process cannot happen in a decentralized / bottom-up fashion: It is often the most junior employees who may have the freshest eyes for obstructive processes - but since they do not know the history of why the processes exists, they often can't effectively propose a change since they don’t know the organisation well enough to rule out unwanted side-effects. This directly sabotages decentralised, bottom-up improvements of workflows.

What is a healthy way to deal with processes?
  1. Realize that they are a form of “organisational memory”: They are often formed as reaction to some unpleasant event - with the intent of preventing this event from repeating. It is also important to realize that unchecked and unchallenged processes can become organisational “scar tissue” - more hindrance than help.
  2. Keep track of the exact motivation for creating each process -- the “why”. This will involve writing half a page or more, and checking with others involved in the creation of the process that the description is accurate and understandable.
  3. The motivations behind the process should be accessible to everybody affected by it.
  4. Everybody should know that company processes are supposed to support, not hinder, getting work done. Everybody should feel empowered to suggest changes in a process - ideally while addressing why these changes will not lead to a repeat of the problem the process was designed to prevent.
  5. People should be empowered to deviate from the process or ignore it - but frequent or even infrequent-but-recurring exceptions are a red flag that the process needs to be improved. Don't accumulate "legacy process" and "organisational debt" through the mechanism of exception-granting.
  6. Everybody should be aware that keeping processes functional and lean is crucial to keeping the organisation healthy. Even if a process is unreasonable and obstructive, most people instinctively try to accept it - but the first instinct should ideally be to change it for the better. Constructively challenging a broken process is a service to the organisation, not an attack.
  7. It may be sensible to treat processes a bit like code - complete with ownership of the relevant process, and version control, and handover of process ownership when people change jobs. Amendments to processes can then be submitted as text, reviewed by the process owner, discussed, and eventually approved - much like a patch or removal of dead code.

Keeping an organisation healthy is hard. The most crucial ingredient to keeping it healthy, though, is that the members of the organisation care to keep it healthy. Therefore it is absolutely critical to encourage fixing the organisation when something is broken - and to not discourage people into "blindly following the process".

August 10 2016


USENIX Security Best Paper 2016 – The Million Key Question … Origins of RSA Public Keys

Petr Svenda et al from Masaryk University in Brno won the Best Paper Award at this year’s USENIX Security Symposium with their paper classifying public RSA keys according to their source.

I really like the simplicity of the original assumption. The starting point of the research was that different crypto/RSA libraries use slightly different elimination methods and “cut-off” thresholds to find suitable prime numbers. They thought these differences should be sufficient to detect a particular cryptographic implementation and all that was needed were public keys. Petr et al confirmed this assumption. The best paper award is a well-deserved recognition as I’ve worked with and followed Petr’s activities closely.

The authors created a method for efficient identification of the source (software library or hardware device) of RSA public keys. It resulted in a classification of keys into more than dozen categories. This classification can be used as a fingerprint that decreases the anonymity of users of Tor and other privacy enhancing mailers or operators.

Bit Length of Largest Prime Factors of p-1 The graphs extracted from: The Million Key Question – Investigating The Origins of RSA Public Keys (follow the link for more).

All that is a result of an analysis of over 60 million freshly generated keys from 22 open- and closed-source libraries and from 16 different smart-cards. While the findings are fairly theoretical, they are demonstrated with a series of easy to understand graphs (see above).

I can’t see an easy way to exploit the results for immediate cyber attacks. However, we started looking into practical applications. There are interesting opportunities for enterprise compliance audits, as the classification only requires access to datasets of public keys – often created as a by-product of internal network vulnerability scanning.

An extended version of the paper is available from http://crcs.cz/rsa.

August 09 2016


IDA 6.95: Qt 5.6.0 configure options & patch

A handful of our users have already requested information regarding the Qt 5.6.0 build, that is shipped with IDA 6.95.

Configure options

Here are the options that were used to build the libraries on:

  • Windows: ...\5.6.0\configure.bat "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "win32-msvc2015" "-opengl" "desktop" "-prefix" "C:/Qt/5.6.0"
    • Note that you will have to build with Visual Studio 2015, to obtain compatible libs
  • Linux: .../5.6.0/configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "linux-g++-32" "-developer-build" "-fontconfig" "-qt-freetype" "-qt-libpng" "-glib" "-qt-xcb" "-dbus" "-qt-sql-sqlite" "-gtkstyle" "-prefix" "/usr/local/Qt/5.6.0"
  • Mac OSX: .../5.6.0/build/../configure "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-opensource" "-force-debug-info" "-platform" "macx-g++-32" "-debug-and-release" "-fontconfig" "-qt-freetype" "-qt-libpng" "-qt-sql-sqlite" "-prefix" "/Users/Shared/Qt/5.6.0"


In addition to the specific configure options, the Qt build that ships with IDA includes the following patch. You should therefore apply it to your own Qt 5.6.0 sources before compiling, in order to obtain similar binaries.

Note that this patch should work without any modification, against the 5.6.0 release as found there. You may have to fiddle with it, if your Qt 5.6.0 sources come from somewhere else.


Installing IDA 6.95 on Linux

IDA is still, as of this writing (August 9th, 2015), a 32-bit application and both IDA & its installer(*) require certain 32-bit libraries to be present on your Linux system before they can run.

Here is the list of commands you will have to run in order to install those dependencies, for the following systems:

  • Debian & derivative systems such as Ubuntu, Xubuntu, …
  • Red Hat Enterprise Linux 7.2 (and likely other versions as well)
Note: we cannot possibly install & try IDA on all flavors/versions of all Linux distributions, but we will do our best to update this post with relevant information, whenever we learn of a distribution requiring special attention.

(*) that is: if you want the installer to run a graphical interface, instead of a command-line one.

Debian & Ubuntu

Common dependencies

The following should allow IDA to run on most Linux systems deriving from Debian distributions:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6-i686:i386 libexpat1:i386 libffi6:i386 libfontconfig1:i386 libfreetype6:i386 libgcc1:i386 libglib2.0-0:i386 libice6:i386 libpcre3:i386 libpng12-0:i386 libsm6:i386 libstdc++6:i386 libuuid1:i386 libx11-6:i386 libxau6:i386 libxcb1:i386 libxdmcp6:i386 libxext6:i386 libxrender1:i386 zlib1g:i386 libx11-xcb1:i386 libdbus-1-3:i386 libxi6:i386 libsm6:i386 libcurl3:i386 

It is necessary to also run the following command, for IDA to present a usable & well-integrated GUI on many Debian & Ubuntu desktops. If the set of dependencies above are not enough to obtain a slick UI, please try the following:

sudo apt-get install libgtk2.0-0:i386 gtk2-engines-murrine:i386 gtk2-engines-pixbuf:i386 libpango1.0-0:i386

Red Hat Enterprise Linux 7.2

IDA will require the following packages to be installed, in order to run properly on RHEL 7.2 (and probably any other RPM-based distribution) :


Arch Linux

While the author of this post is not quite familiar with Arch Linux, one of our users reported that adding the required dependencies can be performed through the AUR package that can be found at: https://aur.archlinux.org/packages/ida-pro-6.4/.

(And then, installing a 32-bit system-wide interpreter (i.e., if one favors that option over the default that consists of using the Python runtime shipped with IDA), should be performed by installing the package ‘lib32-python2’.)

July 29 2016


Yet another Android side channel: input stealing for fun and profit

At PETS 2016 we presented a new side-channel attack in our paper Don’t Interrupt Me While I Type: Inferring Text Entered Through Gesture Typing on Android Keyboards. This was part of Laurent Simon‘s thesis, and won him the runner-up to the best student paper award.

We found that software on your smartphone can infer words you type in other apps by monitoring the aggregate number of context switches and the number of hardware interrupts. These are readable by permissionless apps within the virtual procfs filesystem (mounted under /proc). Three previous research groups had found that other files under procfs support side channels. But the files they used contained information about individual apps– e.g. the file /proc/uid_stat/victimapp/tcp_snd contains the number of bytes sent by “victimapp”. These files are no longer readable in the latest Android version.

We found that the “global” files – those that contain aggregate information about the system – also leak. So a curious app can monitor these global files as a user types on the phone and try to work out the words. We looked at smartphone keyboards that support “gesture typing”: a novel input mechanism democratized by SwiftKey, whereby a user drags their finger from letter to letter to enter words.

This work shows once again how difficult it is to prevent side channels: they come up in all sorts of interesting and unexpected ways. Fortunately, we think there is an easy fix: Google should simply disable access to all procfs files, rather than just the files that leak information about individual apps. Meanwhile, if you’re developing apps for privacy or anonymity, you should be aware that these risks exist.

July 19 2016


PETS 2016

I am at the Privacy Enhancing Technologies Symposium (PETS 2016) in Darmstadt until Friday, and will try to liveblog some of the sessions in followups to this post. (I can’t do them all as there are some parallel sessions.)

March 10 2016


IDA 6.9, Mac OS X, ‘random’ crashes

Intended audience

IDA 6.9 users on Mac OS X, who have suffered seemingly-apparent crashes while using IDA.

The problem

The Qt 5.4.1 libraries shipped with IDA 6.9 suffer from the following bug: https://bugreports.qt.io/browse/QTBUG-44708, which was apparently fixed in Qt 5.5.0.

If, when IDA crashes, you ever spotted a backtrace that looks like the following:

frame #0: 0x00000000
frame #1: 0x00d8a50d QtGui'QT::QTextEngine::shapeText(int) const + 1187
frame #2: 0x00d8b517 QtGui'QT::QTextEngine::shape(int) const + 1199
frame #3: 0x00d8c977 QtGui'QT::QTextEngine::width(int, int) const + 155
frame #4: 0x00d73571 QtGui'QT::QFontMetricsF::width(QT::QString const&) const + 163
frame #5: 0x00041184 idaq'___lldb_unnamed_function853$$idaq + 420

then you’ve been a victim of this rather tiresome issue.

(note: frame #0 doesn’t quite matter; the 2nd line, QT::QTextEngine::shapeText(int), is the important one)

The solution

We have applied the patch mentionned in the Qt bugreport & re-built the libqcocoa.dylib Qt platform support.

You will have to:

  1. download & unzip the following archive: qcocoa.zip.
  2. verify the shasum of the libqcocoa.dylib file:
    $ shasum libqcocoa.dylib 
    afcf3603f593776c6f39f41f81e98843897cf0ed  libqcocoa.dylib
  3. place the libqcocoa.dylib binary instead of the one in /path/to/IDA_6.9/idaq.app/Contents/Plugins/

Once that is done, those crashes shouldn’t happen anymore.

A big, big thank you to Willem Jan Hengeveld & Vladimir Putin, who have reported this!

February 29 2016


Beware: IDA C++ plugins, Qt 5.x, QStringLiteral: crash at exit-time

Intended audience

IDA C++ plugin authors, who wish to link such plugins against Qt 5.x libraries.

The problem

One of our customers, Aliaksandr Trafimchuk, recently reported that whenever IDA was run with a plugin of his that links against the Qt libraries that we ship, IDA would crash at exit-time (at least on Windows.)

Aliaksandr already did most of the work of figuring out exactly what was causing the crash, and even had a work-around (more like a kludge, as he pointed out, really) for it, but he still wanted to let us know about it so we are aware of the problem & perhaps can communicate about it.

The crash is an access violation, in an area of memory that doesn’t seem to be mapped by any stack, heap, DLL code or data.

The stack reveals that the crash happens at QCoreApplication::~QCoreApplication()-time (i.e., at application exit), when the QFontCache is freeing/releasing its entries:

  [External Code] 
  Qt5Gui.dll!QT::QThreadStorage::deleteData(void * x)
  Qt5Core.dll!QT::QThreadStorageData::set(void * p)
  [External Code] 
  idaq.exe!013426c5() Unknown

Why does that happen?

Our customer’s plugin uses a UI description file, that needs to be processed by Qt’s uic (UI-compiler). The generated code contains lines such as these:

label = new QLabel(TestDialog);
QFont font;
font.setFamily(QStringLiteral("Comic Sans MS"));

Note the use of QStringLiteral.

The QStringLiteral type

This is an optimization that came in Qt 5.x, and that causes actual QString instances to be laid out in the .rodata section of the program (together with a special refcount value that is -1, meaning “don’t touch this refcount”.)

Although at exit-time, this “static const” in-.rodata-QString instance wouldn’t be modified (because of the -1 refcount), simply reading it will cause a crash, since the section holding it has been removed from memory.

This is a known limitation/problem, too: https://bugreports.qt.io/browse/QTBUG-46880

The plugin lifecycle

This is where the problem lies: at exit-time, IDA will:

  1. unload plugins
  2. proceed until its QCoreApplication goes out of scope, which will perform (among other things) the QFontCache cleanup.
  3. alas, at that time, the QFontCache still refers to literal QString data, in a section that is now gone (it was discarded at #1)

In fact, Qt expects that any binary that uses Qt libraries should remain in memory, so that some optimizations (such as the QStringLiteral) will continue to work. That’s why, when Qt unloads some of its own plugins, it doesn’t really unload those from memory.

Although the Qt library maintainers consider that having such limitations on binaries that link against Qt is acceptable, I personally hope they try to keep those restrictions as minimal as possible.

The solution

In any case, concerning this QStringLiteral issue, we have a way out: at compilation-time, pass the compiler the following flag: -DQT_NO_UNICODE_LITERAL

This will turn the QStringLiteral() expression into a QString::fromUtf8(), which will allocate the memory on the heap and the plugin should work just fine.

Another possible solution

Another possibility reported by an IDA user (but untested by us), is to add the following after the Qt headers #include directives:

#undef QStringLiteral
#define QStringLiteral(_str) _str

With this method, the literal C-style string will be implicitly converted to a QString, using the default conversion rules.


The kludge (which is Windows-specific) consists of calling LoadLibrary(szMyPluginFilePath), thereby somewhat artificially incrementing the refcount of his plugin, which will cause it to remain in memory & thus the ~QFontCache cleanup will succeed.

January 27 2016


An attempt at fixing Wassenaar

Last year in May, I wrote extensively about the many ways in which the 2013 "intrusion software" amendments to the Wassenaar Arrangement were broken and downright dangerous to all efforts at security the global IT infrastructure. Since then, the debate has heated up from all sides -- extending to a hearing in front of US congress (which was pretty unanimous in condemning these amendments), but also including voices such as James Bamford arguing for these controls in an op-ed. The landscape of the discussion is too complex now to be summarized here (the interested reader can find a partial survey of recent developments here).

Common ground between the different sides of the discussion is not large, but the thing that almost everybody agrees to is "it is bad when despotic regimes that couldn't otherwise get advanced surveillance software purchase sophisticated surveillance software from abroad". How to prevent this is up for discussion, and it is unclear whether export control (and specificially the Wassenaar Arrangement) is the right tool for the task.

To find out whether the export control language can be made to work, my colleagues Mara Tam and Vincenzo Iozzo and me have worked jointly and tried to come up with an amendment to the language of the Wassenaar Arrangement that would satisfy the following criteria:

  • Make sure that click-and-play surveillance frameworks such as the ones marketed by HackingTeam or Gamma are caught and controlled.
  • Make sure that no technology that is required for defending networks (including bugs, proof-of-concept exploits, network scanners etc.) is caught and controlled.
In order to achieve this, we had to depart from the "traditional" Wassenaar language (that is focused on performance metrics and technical properties) and include much greater emphasis on "intent" and especially "informed consent by the user". We draw the line between good and bad if the design intent of the software in question is to be used against people that did not consent.
As of today, we are circulating our draft more widely. We are not 100% sure that our language achieves what we want to achieve, and we are not even sure whether what we want to achieve can be achieved within the language of export control -- but we have made a very thorough effort at testing our language against all scenarios we could come up with, and it worked well.
We are hoping that by circulating our proposal we can somewhat de-polarize the discussion and attempt to find a middle ground that everybody can be happy with -- or, failing that, to show that even with a lot of effort the 2013 amendments may end up being unfixable.
Anyhow, if you are interested in our document, you can download it here. As we get more feedback, the document will be updated and replaced with newer versions.

December 30 2015


IDAPython: migrating PySide code to PyQt5


Contrary to previous versions that shipped with Qt 4.8.4, IDA 6.9 ships with Qt 5.4.1 and as we announced some time ago, this will force some changes for plugin writers who were using IDAPython + PySide in order to build custom interfaces.

What’s more, in addition to the Qt4 -> Qt5 switch, we have also chosen to drop PySide in favor of PyQt5, as it seems to be more actively developed.


While we were porting some internal plugins & test scripts, we have come across a number of ‘issues’ that are due both to the change in major Qt version, and also to the switch to PyQt5.

Here is a (non-exhaustive) list of such problems that needed to be dealt with.

Use ‘from PyQt5’ instead of ‘from PySide’

   from PySide import QtCore, QtGui


   from PyQt5 import QtCore, QtGui, QtWidgets

(notice the added ‘QtWidgets’. Keep reading.)

Most widgets are now in QtWidgets, not QtGui anymore (Qt5 architecture change)

   from PySide import QtGui
   edit = QtGui.QTextEdit()


   from PyQt5 import QtWidgets
   edit = QtWidgets.QTextEdit()

Converting TForm instances must be done to PyQt

   from PySide import QtGui, QtCore
   form = idaapi.find_tform("Output window")
   w = idaapi.PluginForm.FormToPySideWidget(form)


   from PyQt5 import QtGui, QtCore, QtWidgets
   form = idaapi.find_tform("Output window")
   w = idaapi.PluginForm.FormToPyQtWidget(form)

Public QEvent types are in QtCore.QEvent namespace




Using QObject.connect() to connect signal with slot doesn’t work anymore. Use alternative approach.

   obj.connect(query_action, QtCore.SIGNAL("triggered()"), self.ui.queryGraph)



Misc. required changes (reported by our users)

  • QHeaderView.setResizeMode => QHeaderView.setSectionResizeMode
  • QtCore.Qt.ItemFlag enum was moved to be accessed directly on QtCore.Qt
  • QColorDialog.setCustomColor now takes a QColor instead of an RGB int as the second argument.

Final words

As I said above, this is a non-exhaustive list of issues we encountered. Should you encounter other issues, please let us know about them, so we can update & improve this page to help the community!

December 29 2015


IDA 6.9: Qt 5.4.1 configure options & patch

A handful of our users have already requested information regarding the Qt 5.4.1 build, that is shipped with IDA 6.9.

Configure options

Here are the options that were used to build the libraries on:

  • Windows: ...\5.4.1\configure.bat "-debug-and-release" "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-platform" "win32-msvc2015" "-opengl" "desktop" "-force-debug-info" "-prefix" "C:/Qt/5.4.1"
    • Note that you will have to build with Visual Studio 2015, to obtain compatible libs
  • Linux: .../5.4.1/configure "-debug-and-release" "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-platform" "linux-g++-32" "-developer-build" "-fontconfig" "-qt-freetype" "-qt-libpng" "-glib" "-qt-xcb" "-dbus" "-qt-sql-sqlite" "-gtkstyle" "-prefix" "/usr/local/Qt/5.4.1"
  • Mac OSX: .../5.4.1/configure "-debug-and-release" "-nomake" "tests" "-qtnamespace" "QT" "-confirm-license" "-accessibility" "-platform" "macx-g++-32" "-fontconfig" "-qt-freetype" "-qt-libpng" "-qt-sql-sqlite" "-prefix" "/Users/Shared/Qt/5.4.1"


In addition to the specific configure options, the Qt build that ships with IDA includes the following patch. You should therefore apply it to your own Qt 5.4.1 sources before compiling, in order to obtain similar binaries.

December 23 2015


Installing IDA 6.9 on Linux

IDA is still, as of this writing (December 23rd, 2015), a 32-bit application and both IDA & its installer(*) require certain 32-bit libraries to be present on your Linux system before they can run.

Here is the list of commands you will have to run in order to install those dependencies, for the following systems:

  • Debian & derivative systems such as Ubuntu, Xubuntu, …
  • Red Hat Enterprise Linux 7.2 (and likely other versions as well)
Note: we cannot possibly install & try IDA on all flavors/versions of all Linux distributions, but we will do our best to update this post with relevant information, whenever we learn of a distribution requiring special attention.

(*) that is: if you want the installer to run a graphical interface, instead of a command-line one.

Debian & Ubuntu

Common dependencies

The following should allow IDA to run on most Linux systems deriving from Debian distributions:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6-i686:i386 libexpat1:i386 libffi6:i386 libfontconfig1:i386 libfreetype6:i386 libgcc1:i386 libglib2.0-0:i386 libice6:i386 libpcre3:i386 libpng12-0:i386 libsm6:i386 libstdc++6:i386 libuuid1:i386 libx11-6:i386 libxau6:i386 libxcb1:i386 libxdmcp6:i386 libxext6:i386 libxrender1:i386 zlib1g:i386 libx11-xcb1:i386 libdbus-1-3:i386 libxi6:i386 libsm6:i386 libcurl3:i386 

Xubuntu 15.10

It is necessary to also run those commands, for IDA to present a usable GUI on Xubuntu 15.10

sudo apt-get install libgtk2.0-0:i386 gtk2-engines-murrine:i386 gtk2-engines-pixbuf:i386 

Red Hat Enterprise Linux 7.2

IDA will require the following packages to be installed, in order to run properly on RHEL 7.2 (and probably any other RPM-based distribution) :


December 22 2015


Open-Source BinNavi ... and fREedom

One of the cool things that my former zynamics colleagues (now at Google) did was the open-sourcing of BinNavi - a tool that I used to blog about quite frequently in the old days (here for example when it came to debugging old ScreenOS devices, or here for much more - kernel debugging, REIL etc.).

BinNavi is a GUI / IDE for performing multi-user reverse engineering, debugging, and code analysis. BinNavi allows the interactive exploration and annotation of disassemblies, displayed as browsable, clickable, and searchable graphs - based on a disassembly read from a PostgreSQL database, which can, in theory, be written by any other engine.

Writing UIs is hard work, and while there are many very impressive open-source reverse engineering tools around (Radare comes to mind first, but there are many others), the UI is often not very pretty - or convenient. My hope is that BinNavi can become the "default UI" to a plethora of open-source reverse engineering tools, and grow to realize it's full potential as "the open-source reverse engineering IDE".

One of the biggest obstacles to BinNavi becoming more widely adopted is the fact that IDA is the only "data source" for BinNavi - e.g. while BinNavi is FOSS, somebody that wishes to start reverse engineering still needs IDA to fill the Postgres database with disassembly.

To remedy this situation, Dave Aitel put up a contest: Anybody that either builds a Capstone-to-BinNavi-SQL-bridge or that adds decompilation as a feature to BinNavi gets free tickets to INFILTRATE 2016.

Last week Chris Eagle published fREedom, a Python-based tool to disassemble x86 and x86_64 programs in the form of PE32, PE32+, and ELF files. This is pretty awesome - because it means that BinNavi moves much closer to being usable without any non-free tools.

In this blog post, I will post some first impressions, observations, and screenshots of fREedom in action.

My first test file is putty.exe (91b21fffe934d856c43e35a388c78fccce7471ea) - a relatively small Win32 PE file, with about ~1800 functions when disassembled in IDA.

Let's look at the first function:
Left: IDA's disassembly output. Right: fREedom's disassembly output So disassembly, CFG building etc. has worked nicely. Multi-user commenting works as expected, as does translation to REIL. Callgraph browsing works, too:

The great thing about having fREedom to start from is that further improvements can be incremental and layered - people have something good to work from now :-) So what is missing / needs to come next? 
  1. fREedom: Function entry point recognition is still relatively poor - out of the ~1800 functions that IDA recognizes in putty, only 430 or so are found. This seems like an excellent target for one of those classical "using Python and some machine learning to do XYZ" blog posts.
  2. fREedom: The CFG reconstruction and disassembly needs to be put through it's paces on big and harder executables.
  3. BinNavi: Stack frame information should be reconstructed - but not by fREedom, but within BinNavi (and via REIL). This will require digging into (and documenting) the powerful-but-obscure type system design.
  4. BinNavi: There has been some bitrot in many areas of BinNavi since 2011 - platforms change, systems change, and there are quite some areas that are somewhat broken or need updating (for example debugging on x64 etc.). Time to brush off the dust :-)
Personally, I am both super happy and pretty psyched about fREedom + BinNavi, and I hope that the two can be fully integrated so that BinNavi always has fREedom as default disassembly backend.

December 16 2015


A decisionmaker's guide to buying security appliances and gateways

With the prevalence of targeted "APT-style" attacks and the business risks of data breaches reaching the board level, the market for "security appliances" is as hot as it has ever been. Many organisations feel the need to beef up their security - and vendors of security appliances offer a plethora of content-inspection / email-security / anti-APT appliances, along with glossy marketing brochures full of impressive-sounding claims.

Decisionmakers often compare the offerings on criteria such as easy integration with existing systems, manageability, false-positive-rate etc. Unfortunately, they often don't have enough data to answer the question "will installing this appliance make my network more or less secure?".

Most security appliances are Linux-based, and use a rather large number of open-source libraries to parse the untrusted data stream which they are inspecting. These libraries, along with the proprietary code by the vendor, form the "attack surface" of the appliance, e.g. the code that is exposed to an outside attacker looking to attack the appliance. All security appliances require a privileged position on the network - a position where all or most incoming and outgoing traffic can be seen. This means that vulnerabilities within security appliances give an attacker a particularly privileged position - and implies that the security of the appliance itself is rather important.

Installing an insecure appliance will make your network less secure instead of safer. If best engineering practices are not followed by the vendor, a mistake in any of the libraries parsing the incoming data will compromise the entire appliance.

How can you decide whether an appliance is secure or not? Performing an in-depth third-party security assessment of the appliance may be impractical for financial, legal, and organisational reasons.

Five questions to ask the vendor of a security appliance

In the absence of such an assessment, there are a few questions you should ask the vendor prior to making a purchasing decision:

  1. What third-party libraries interact directly with the incoming data, and what are the processes to react to security issues published in these libraries?
  2. Are all these third-party libraries sandboxed in a sandbox that is recognized as industry-standard? The sandbox Google uses in Chrome and Adobe uses in Acrobat Reader is open-source and has undergone a lot of scrutiny, so have the isolation features of KVM and qemu. Are any third-party libraries running outside of a sandbox or an internal virtualization environment? If so, why, and what is the timeline to address this?
  3. How much of the proprietary code which directly interacts with the incoming data runs outside of a sandbox? To what extent has this code been security-reviewed?
  4. Is the vendor willing to provide a hard disk image for a basic assessment by a third-party security consultancy? Misconfigured permissions that allow privilege escalation happen all-too often, so basic permissions lockdown should have happened on the appliance.
  5. In the case of a breach in your company, what is the process through which your forensics team can acquire memory images and hard disk images from the appliance?
A vendor that takes their product quality (and hence your data security) seriously will be able to answer these questions, and will be able to confidently state that all third-party parsers and a large fraction of their proprietary code runs sandboxed or virtualized, and that the configuration of the machine has been reasonably locked down - and will be willing to provide evidence for this (for example a disk image or virtual appliance along with permission to inspect).

Why am I qualified to write this?

From 2004 to 2011 I was CEO of a security company called zynamics that was acquired by Google in 2011. Among other things, we used to sell a security appliance that inspected untrusted malware. I know the technical side involved with building such an appliance, and I understand the business needs of both customers and vendors. I also know quite a bit about the process of finding and exploiting vulnerabilities, having worked in that area since 2000.
Our appliance at the time was Debian-based - and the complex processing of incoming malware happened inside either memory-safe languages or inside a locked-down virtualized environment (emulator), inside a reasonably locked-down Linux machine. This does not mean that we never had security issues (we had XSS problems at one point where strings extracted from the malware could be used to inject into the Web UI etc.) - but we made a reasonable effort to adhere to best engineering practices available to keep the box secure. Security problems happen, but mitigating their impact is not rocket science - good, robust, and free software exists that can sandbox code, and the engineering effort to implement such mitigations is not excessive.

Bonus questions for particularly good vendors

If your vendor can answer the 5 questions above in a satisfactory way, his performance is already head-and-shoulders above the industry average. If you wish to further encourage the vendor to be proactive about your data security, you can ask the following "bonus questions":
  1. Has the vendor considered moving the Linux on their appliance to GRSec in order to make privilege escalations harder?
  2. Does the vendor publish hashes of the packages they install on the appliance so in case of a forensic investigation it is easy to verify that the attacker has not replaced some?

December 08 2015


IDA + Windows + system python 2.7.11

In case you:

  • are running IDA on Windows
  • are using the system’s Python (as opposed to the bundled Python distribution, that one can opt for at installation-time)
  • have installed the recently released Python 2.7.11

…you will have noticed that IDAPython fails to load the ‘site’ module and, consequently, IDAPython is not available.

As far as I understand, this is a Python installer issue, which I reported ( https://bugs.python.org/issue25824 )

While waiting for feedback from the Python authors, we can already offer the two following solutions:

  1. remove Python 2.7.11 and pick an earlier version (2.7.10 is known to work)
  2. open regedit.exe, and rename the key:
    HKLM\Software\Wow6432Node\Python\PythonCore\2.7\PythonPath into

This post will be updated whenever more information becomes available on the Python side.


(Thanks to Tamir Bahar, who reported this issue very early on!)

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!