When will Google’s search engine monopoly be broken?

When will Google’s search engine monopoly be broken?

In the 1990s I was an academic at the University of Edinburgh teaching electronics and computing. In 1998 a colleague introduced me to the Google search engine and as a result I made the shift from my previously preferred search engine, Alta Vista, to Google. Twenty years on, despite trying many others, I still use Google regularly. I was delighted by the Google experience 20 years ago, but today I dislike Google search for several reasons but I have yet to find a noticeably better alternative.

In a recent poll conducted by the Search Engine Journal (SEJ) it was revealed that that Duck Duck Go (DDG) is the most popular alternative to Google (based on 1097 SEJ Twitter followers who responded). However, according to Statcounter DDG have just 0.32% (October 2018) of the search market and Google have 92.74%. My question is whether Google’s monopoly could be under threat by emerging search engines such as DDG and if so, why have they not yet made a major impact on Google’s market share?


Google has held a monopoly position for around 20 years and in that time ‘to Google’ has become a verb. This incumbent position will not be easy to shift. DDG has a very similar product to Google it has not made much of a dent in the market. Their major differentiator is that it respects user privacy. However, if we are honest it does not deliver truly better search results, in fact in my experience the results are generally less good – and are still tainted by significant volumes of undesirable advertising content. Thus, DDGs market is a niche market segment that believe user privacy is essential and are willing to accept slightly inferior search results to achieve this. This market sector (which I include myself in) is huge in world terms – but remains a niche.

My shift from Alta Vista to Google some 20 years ago was based on it being a better solution to my internet search problem — data privacy was not generally thought about in 1998. For me DDG is not yet a better solution, if better results and less advertising (which I know is how they make money) was delivered alonside its enhanced privacy, then it could be a winner. The other problem for DDG is that the federated results use Bing as a major source of data, so they do not have an independent platform from which they can base an assault on Google.

My current opinion is that Google will slowly lose market share to a variety of small search engines such as DDG. One of these players will emerge as a future challenger to Google within the next 5 years and we may not even have heard of them yet. The successful search engine will likely do their own crawling and indexing so they are not dependent on Bing, Google or Yahoo and they will not rely on pay-per-click (PPC) advertising which most users dislike. It is quite probable that when they do emerge as a leading contender they will be acquired and scaled by a company with a large existing subscriber base and complimentary business activities such as Amazon.

Avoiding the Filter Bubble

Avoiding the Filter Bubble

For those not familiar with the ‘Filter Bubble’, the term was coined by Internet activist Eli Pariser around 2010 and refers to a state of intellectual isolation resulting from personalisation applied to the delivery of web content. The suggestion is that users become separated from information that disagrees with their viewpoints. This effectively isolates individuals within their own cultural or ideological filter bubble.

While we could debate at length the extent to which this is a problem, few would deny its existence. In his 2011 TED Talk Eli Pariser suggests that search engines should curate results according to the following criteria, not just by relevance:

Sort by

I personally don’t fully subscribe to this view. It requires that the search algorithms will somehow curate the results according to these ethics. Thus, we have to trust someone to decide what ethical parameters are good for us and code these into the algorithms! Or, of course we could go the other way and completely remove personalisation from any results and some search engines, such as Duck Duck Go, do not personalise results and so offer us this option. However, we then lose the ability to have more relevant results ranked higher and so more manual sifting of results by the user is needed. My view is that the Filter Bubble can be largely avoided if we provide visibility of the personalisation process and control over how our personal data is being applied.

Intellectual isolation resulting from the delivery of personalised content is clearly not peculiar to the Internet. In the western world we have the choices everyday of who to listen to, what to read, the media channels to watch and the ability to switch these on and off at will. If one chooses only to read or listen to right-wing views, then this is a personal right. Each individual is at least aware that not everyone shares these same views, and they can see that alternative reading and viewing material is available. Where Internet personalisation has a real problem is that this isolation can happen (indeed is happening) without there being visibility or self-awareness of it. It becomes especially dangerous if it is used to alter opinions for financial or political gain, and we are now aware of previous abuses of influence in recent elections and referendums.

I believe that users should be given visibility of the data they are revealing for personalisation. This transparency gives the ability for self-observation (see your digital-self as revealed to others), self-measurements (revealing of statistics, trends and actionable insights), and self-comparisons (how you measure up relative to your peers). This can theoretically be achieved using existing rights to data portability under GDPR and would be a useful first step in avoiding the Filter Bubble.

The next step would then be the ability to give the user control over the degree of personalisation delivered by any service. The user regulates the cost-benefit of revealing personal information and sees how the quality of content delivered is being influenced by this. In reality, if someone wishes to live in a state of intellectual isolation this should be a choice they have made, not a situation that they are unaware of or that has been imposed upon them.

The California Consumer Privacy Act – CCPA

The California Consumer Privacy Act – CCPA

Oh no, more data privacy legislation is coming! The California Consumer Privacy Act (CCPA) will kick in on 1st January 2020. For many of us the key questions are; what is the difference between CCPA and GDPR, and will it affect me?

The short answers are that it has similarities to GDPR but is different in several aspects, and secondly – in theory it should only really affect you if you are a citizen of California, or if you are a large company that processes personal information that includes one or more citizens from California. However, its reach will be much broader than this since companies are unlikely to adopt different policies exclusively for their Californian customers.

I am going to elaborate on these points, but still want to keep it short and so have generalised a little – you can read about it in more detail from the links I will give at the bottom.

How will CCPA affect us?

Most individual citizens may not notice much impact from CCPA, even as it comes near to the enforcement date. We may see a small email storm but less than we suffered for GDPR. The reason for this is that CCPA is largely based on opt-out rather than GDPR’s opt-in.  American citizens (especially those in California) are likely to see more activity prior to implementation. For most of us, we will be told that service terms & conditions are being updated, but then this happens regularly anyway and few of us read these before agreeing.

After implementation one change we may observe is the appearance of opt-outs regarding the sale of data – a “Do Not Sell My Personal Information” option as default – you will need to select this in order to opt-out. In addition, those aged 13-16 should automatically be opted out and so “I am over 16” might also be ticked as a default. Those under 13 will require parental consent to allow their data to be sold. Interestingly a company could sell the information of a minor without consent if they do not have “actual knowledge” that the consumer is under 16 so it will be interesting to see how this will be interpreted in practice.

In the short-term it is companies with Californian customers that will be most affected. However, the Act only applies to companies of a certain size (turnover of $25 million or more), or that hold personal information on more than 50,000 consumers (or devices). Thus, it will be large companies, most of whom will already have dealt with GDPR compliance, that will be most affected. It is likely that these companies will develop policies for customers that jointly covers GDPR and CCPA.

So, will citizens benefit from CCPA? If we are already under GDPR then I can see little if any benefit. If you are subscriber to services that do not need to be GDPR compliant (e.g. a US citizen who’s personal information is held in the USA) then CCPA can give you GDPR-like rights and benefits regarding the use of your Personal Information. However, CCPA is not the same as GDPR and in many ways it is weaker.

The main differences between CCPA and GDPR

Who it applies to: CCPA only applies to citizens of California, and of course it will apply to companies that hold personal information on any citizen of California and so it will have global impact. GDPR applies to companies operating in the EU, it also applies to companies that hold or process data on EU citizens so GDPR has a greater global impact.

What does it apply to: The Act applies to “personal information”. The definition is quite wide and similar to GDPR but slightly wider in the sense that if “households” can be identified from the data then it is considered “personal information” even although an individual is not identified.

Who is regulated: CCPA only applies to larger for-profit companies that process or hold significant “personal information”. The company will have a turnover of above $25 million; or process/hold personal information of 50,000 or more individuals, households or devices; or make more than 50% of revenues from selling personal information. Any business that is not in California and does not use information on the states citizens is completely exempt. GDPR on the other hand applies to almost all companies (no matter the size) that use personal information in the EU or related to EU citizens.

What rights does it give: Consumers protected by CCPA are entitles to be given notice about the categories of information being collected and the business purpose for which it is being collected, plus any intention to sell this information with the option to opt-out of this sale. Note that CCPA relies on an opt-out policy, whereas GDPR is opt-in. The customer has the right to be told what type of information is being held, although in practice this might be a boilerplate list. The customer may request that their personal information is deleted (and there are a few exemptions to this) so this is similar to, but not quite the same as, GDPR’s ‘right to be forgotten’. CCPA has no direct equivalent to GDPR’s data portability i.e. the right to request a copy of your personal information. To comply a company only needs to disclose information about what has been collected over the last 12 months but the Act does not seem to provide an explicit right to obtain a full copy of the actual data itself. All customers must be treated equally under the Act meaning that a request made under the Act cannot be used as a reason to alter any terms or pricing for that customer.

Penalties:  Under CCPA damages of $100-750 per consumer per incident are applicable. For GDPR the penalties can be Up to €20 million, or 4% annual global turnover – whichever is higher. So, although different, both can apply severe penalties.

Read more …

This has been a very superficial look at CCPA, but hopefully its conciseness makes it useful and readable. Here are a couple of articles that provide more detailed information that I found useful in trying to understand CCPA.

CCPA: What Marketers Need to Know about the California Consumer Privacy Act

CCPA and GDPR: Comparison of certain provisions

Yelling at strangers all day trying to make a living?

Yelling at strangers all day trying to make a living?

Is the internet really broken? Are the days of direct on-line advertising numbered? Could there be a better way to pay for internet services? Are data marketplaces the answer?

There are companies emerging in the personal data space that propose data marketplaces as a revenue generating model – and they have collectively raised a lot of investment. The model assumes you are willing to share your personal data, they will sell your data for you – you get paid for this and, of course, these companies get a cut.  The argument goes along the lines of, why should someone like Google get all the advertising revenues from your data. Instead you can use a third party to sell your personal data to advertisers and organisations in order to get a small payback.

Some of these companies claim that “The Internet is Broken” and this is a way to take control of your personal data and fix this. I don’t really buy that, I do believe that the current Internet has significant flaws, in part caused by the ‘free’ services model we have come to expect. However, the idea that getting paid to be advertised to is a potential solution, does not add up for me. There is evidence that we are unhappy with the current advertising model, that we would like improved internet services, but also that we are not willing to pay directly for most services that are currently seen as ‘free’.

Advertising is a very inefficient model, how many impressions of an advert result in an actual lead or a sale? There are three parties involved in the traditional on-line advertising model but only one of them gets a significant benefit – the adverting company. The seller and the consumer seem to get the poor end of the bargain but have limited powers due to a lack of competition. The consumer gets constantly bombarded with unwanted advertisements, the seller has to pay the cost of this inefficiency in order to gain on-line exposure to customers. An oligopoly exists in on-line advertising with Google and Facebook owning over 60% of the worldwide market and these companies hold monopoly positions with search and social media respectively. These ‘free’ internet services are the source from which our personally identifiable information (PII) is harvested in order to power the ad machine.

In a 2009 blog Seth Godin, the pioneer of permission marketing, was repeating a point about not needing many customers, but needing good loyal customers, to make a living, and he talked about “yelling at strangers all day trying to make a living” which pretty much sums up the current on-line advertising model, and he went on to say “You don’t find customers for your products. You find products for your customers.” which begins to describe an on-line brokerage model that I believe can be used to erode the on-line advertising model. Displacement of the current model will not happen overnight for a number of reasons; first – the on-line advertising oligopolies will defend their current lucrative position, secondly – although the model might be more efficient in the long run it will introduce some inital pain for the seller adjusting to a new model, and finally – it requires the creation of new services to efficiently join up buyers and sellers.

BrokerageThere are many examples of very successful brokerage companies working in niche areas. In the UK two of the largest online companies Money Supermarket and Skyscanner offer services that find their customers what they are looking for efficiently. Worldwide; Ebay, Uber, Airbnb & Kayak are all examples of highly successful companies running efficient brokerage models. Consider the conversion rate and value of a visitor to one of these sites compared to on-line advertising. It has been estimated that the average citizen is exposed to more than 5000 adverts every single day (J Walker-Smith, Yankovich Inc.), so how can you possibly gain attention in such a marketplace without a strong brand and a lot of marketing budget?  Unfortunately, if you are a small company playing this game you may well spend all day yelling at strangers, and may not manage to make a living, even if you do have the best product.

Once we can capture rich data on the client-side we can begin applying brokerage, affiliate and referral models that will benefit the citizen across, not just niche areas, but all products and services. This will provide more efficient sales channels to sellers and ultimately erode the advertising revenues of the existing oligopolies. Machine learning and AI can be much more effective on the client-side where we can trust our that our personal data is only being used by digital agents working to our agenda (we may even use blockchain to make this transparent), as opposed to the current secret agents working to their own agenda.