Archive for the ‘data’ tag
Posted by randfish
If you've used or considered using the Mozscape API to retrieve link metrics data, we've got something unique to share – a brand new beta of a much faster, more robust API. This beta version currently has just a few testers (and we're seeking more), but thus far, we're seeing remarkable results.
Carin, who manages the big data team here at Moz, helped share the story with me last week:
- The current API is not able to support everyone's use case! Some people need to make a lot of calls in a really short period of time – our API currently can't support more than 10 requests/second (even for paid users). Others have a large list of URLs they want to update metrics on every new index release – our current API doesn't support batching very well and will timeout with batch sizes larger than 50 URLs.
- The beta version has made some serious performance improvements with single URL throughput and can handle 200 requests / second – the beta API is seeing a 400% throughput improvement, although response times will still be the same
To address batching users we've developed a new batching model – online batching (available in the beta API) and offline batching (coming soon to the beta API)
- Online batching: the maximum amount of results we can process in a POST without a timeout from S3. This has been improved from 50 URLs to 500 URLs in one batch request
- Offline batching (still in development): for batch sizes larger than 500 URLs, offline batching will process through the entire list (probably up to a certain limit not yet decided) and return a downloadable CSV link to S3 where all the data will be available. Since this is still in development, it is not clear the SLA on offline batching, but this feature will be also be available for beta testing as soon as it is feature complete!
Mozscape's API is pretty big today – we served 154,352,249 (154 million) requests in the first 10 days of August and returned 1,186,736,774 (1.2 billion) rows of link metrics data.
You can still sign up free or try our paid API, but if you have serious demand for high-volume or large batches of link data, we'd love to have you in the beta for the new API. Just contact Andrew Dumont – email@example.com – and he'll get you set up!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!
Google may be taking the ill-conceived Nexus Q back to the drawing board, but the Nexus 7 tablet has been quite a hit since it went on sale a few weeks ago. Since then, though, Google has faced a number of supply issues and the company even suspended sales of the 16GB for a few days to catch up with demand. The latest data from ad network Chitika shows how those supply issues have stalled the growth of Google’s first tablet – at least when measured by web traffic from Nexus 7 owners.
Just a few weeks ago, Chitika reported that the Nexus tablet was on its way to surpass the Kindle Fire in its traffic rankings. By now, the company predicted, the Google tablet was supposed to be ahead of the Fire. Instead, its latest data shows that it’s still trailing Amazon’s tablet by quite a bit. While traffic from the Nexus 7 to Chitika’s network of member sites grew rapidly during the first few weeks after it went on sale, growth stalled over the last three weeks. Chitika reports its tablet data relative to the iPad and the Nexus 7 currently accounts for 0.35 impressions per 100 iPad impressions. That’s virtually unchanged from the 0.3 impressions it reported three weeks ago.
With only a few new Nexus 7 owners surfing the web due to Google’s supply issues, the company’s numbers probably reflect the usage of existing users. ”While the Nexus 7 experienced a huge initial surge in both sales and Web traffic, users don’t seem to be surfing as much as one might expect them to.” As the novelty of the device wears off and as the honeymoon phase comes to an end, people simply don’t use it as much as they used to. Now that the Nexus 7 is back on sale, it will be interesting to see if the growth rate picks up again.
You have likely heard something about “big data” and wondered what it is and if it has any impact on marketers and your world.
There are lots of ways people use (and misuse) the term, but for purposes of this column, let’s say that big data is marketing data that is multistructured (not linear or easily aligned to a structured database format) and sourced from multiple customer interactions. That might include clickstream (website visits), behavioral insights, email and SMS response data, social posts, tweets, and search keyword activity.
In essence, big data disrupts marketing. It upsets the normal “container” of marketing data because the unstructured and multistructured formats don’t match the kinds of one-to-one relationships of data element with database field (the way that structured data works). Big data upsets the CRM paradigm because it’s fluid, hard to sort and prioritize, and not always attributable to a specific person. Big data also disrupts the infrastructure capex budget.
Big data is just that: big.
Gartner has reported that competitive advantage goes to folks who tap into this disruption of data.
Do not be afraid. There are plenty of opportunities that involve harnessing big data and making sense of it. At one level, it’s important to just ask questions of the data. You can only make better decisions if you use the gems hidden in your vast data storehouses. Better yet, imagine what you could do if you could use all the data you have. And I mean: All. The. Data. That is pretty exciting. You’d be doing things like social community relationship analysis, persona-based segmentations, behavioral modeling, path-to-purchase analysis, real-time offer management, multi-touch attribution analysis, advertising and media analysis, and more.
Big data can be an incredible opportunity, but folks can be frustrated by trying to get their arms around it.
One area where marketers are optimizing their investments in big data analysis is in the area of digital marketing attribution, which is itself the first step to digital marketing optimization. Most attribution today is last click, more for the complexity in managing data than from marketer choice. But now that we are tapping big data, attribution analysis can track behavioral insights and better understand and serve customers who are interacting across an expanding universe of multiple channels, touchpoints, and data sources—everything from email to search, digital advertising, websites, and social media.
The volume and complexity of new data sources require advanced analytics beyond “last touch” or “last click” attribution. To make accurate budgeting decisions, marketers need to take into account multichannel, multitouch purchasing cycles.
Consider two examples of how attribution could work for you.
- A major online and offline retailer leverages big data to derive consumer insights that are deployed across channels. Instead of relying on sampling, customer intelligence is created from big data analysis. Customers benefit from more personalized experiences.
- An online-only retailer ties together clickstream information with email logs, ad viewing information, and operational information to identify customer preferences and behavior—and to optimize marketing spend. That includes parsing of Twitter feeds and sentiment analysis.
Data-driven marketers must think differently. Our customers expect it, and our markets demand it.
Consider these types of initiatives for your own organization, where digital marketing attribution can help.
- Gain visibility into marketing activity to optimize the use of new channels and deliver remarkable customer experience across conversation points.
- Automate marketing processes and simplify cross-channel measurements. Facilitate experimentation and iteration to optimize digital channels by using quantitative results as they happen.
- Empower yourself to think differently; think that the answer is already there in the data.
- Take a complete look at the data, both digital and non-digital information, to get a more complete view of customers, their preferred channels, and interactive behavior.
- Procure the right big data analytics tools that will integrate with your marketing database, campaign management, CRM, and digital messaging solutions.
What is your story around attribution? Are you on a path to tap the disruption of data or are you sticking with last click attribution models?
Share your learnings below.
(Photo courtesy of Bigstock: Comical Boy)
Google announced they’ve expanded their traffic data to over 130 smaller cities across the world.
The expansion is not just in the US, but also in places like Colombia, Costa Rica, and Panama…
Google announced they’ve expanded their traffic data to over 130 smaller cities across the world.
The expansion is not just in the US, but also in places like Colombia, Costa Rica, and Panama. Google also added cities in the US such as Kalamazoo (Michigan), Portland (Maine), Tuscaloosa (Alabama) .
Personally, I love Google Maps traffic and with Apple Maps replacing Google Maps in iOS 6, despite Apple Maps having real time traffic, I am a bit nervous about it’s accuracy. I love being able to see how much longer that red line in Google Maps is when sitting in bumper to bumper traffic in New York. So expanding Google Maps real time traffic to more and more cities is great!
As you know, Google Maps crowdsources most of the traffic data, so it is pretty good.
Here is a coverage map of traffic data covered by Google Maps.
We talk a lot about analytics.
We collect a lot of data.
We know that there is business value in data.
If we know all of this why is it that marketers are finding it so hard to truly use analytics?
There are many reasons and a recent study by eConsultancy and Lynchpin (repackaged by eMarketer) gave some of them. The first relates to the sheer volume and data and finding how much is actually useful to marketers.
Over 50% of the respondents said that only 50% of the data collected is useful to their business. So why is it being collected? Good question. One way that marketers can make their lives easier is to collect only the pertinent data rather than collecting everything under the sun and thus creating an opportunity to miss valuable in formation in the clutter.
The next finding is interesting as well. Nearly 50% of those spoken too either did not have a business intelligence strategy OR didn’t integrate web analytics with larger business intelligence efforts at all. Again we have to ask, ‘Why is it being collected then?”
These questions are tough ones for marketers. It could be that many marketers are not statisticians or are not as mathematically inclined as today’s data driven world demands. These are skills that can be acquired but getting someone to take the time and effort to learn is a different matter.
So what does the marketing industry as a whole need to do? It needs to align the geeks with the creatives. Creatives get nervous around piles of numbers that are supposed to prove something. Creatives are about messaging and the psychological aspects of the marketing game. It’s almost like the void that exists between engineers and sales people where you need a sales engineer to bridge the gap. Do we need to develop marketing engineers as go betweens?
Anything is better than seeing these statistics which show that the collecting of data far outpaces the application of data. This kind of activity will be the thing that slows progress in the Internet space.
What do you suggest we do about it?
After months of hype, it’s been pretty quiet around Pinterest for the last few weeks. According to the latest data we just received from Shareaholic, however, the social photo sharing site is still growing quickly, though judging from the company’s referral data, Pinterest’s growth has slowed down a bit since June. From May to June Pinterest’s referral traffic to Shareaholic’s network of about 200,000 sites grew by 43.7%. From June to July, it “only” increased by 15.97%. Even at this slower growth rate, Pinterest is still on track to surpass referral traffic from Yahoo in August. This, says Shareaholic, means Pinterest will soon be the fourth largest traffic source in the world.
Today, Pinterest already drives more traffic to sites that use Shareaholic than Bing and Twitter and it’s only 0.10% away from overtaking Yahoo. Google, of course, remains the largest driver of traffic to the company’s member sites, followed by Facebook and – for the time being – Yahoo.
As our own Sarah Perez reported a few weeks ago, it was only in June that Pinterest passed Bing, Twitter and Stumbleupon in the company’s rankings.
Google+, by the way, doesn’t even appear in Shareaholic’s graphs, but as a company spokesperson told me earlier today, it accounted for a minuscule 0.06% of its referral traffic last month.
Shareaholic says its data is based on referral traffic to more than 200,000 publishers who reach more than 270 million people each month. It’s important to keep in mind that this data doesn’t necessarily reflect the Internet as a whole, but the company’s data has generally been quite similar to what we’ve seen from other third-party analytics services.
Your computer is no longer the only device with which you access the internet. In fact, studies are showing that it’s not even the most common device you use to access the internet. And when even an internet connection is no longer a constant, there’s barely any line left between web development and native development for most agencies.
As we’re trying to transition through this sea change — arguably the most significant shift our industry has seen since the internet itself became a household fixture — each step takes us into new terrain. And we’re not all moving at the same pace. Some of us are walking, others running. The tech companies seem to have built themselves jetpacks. In fact, Rey Interactive CTO Kevin Cogill believes they are powered by arc reactors.
In theory, HTML5 is the new sheriff in town. It has the potential to become a common denominator that can provide rich experiences across a wide variety of devices. But even as we’re finally starting to understand and define HTML5 actually is, it can’t do what a native iOS or Android app can do, nor is it in any position quite yet to unseat Flash as the backbone of the banner advertising industry. And then there’s Facebook, which is essentially splintered into two different experiences, and no one’s quite sure if or how Zuckerberg will be able to bring all of Facebook’s third-party platform apps onto mobile.
Whilst reminiscing about the good old days — when things were simpler — Kevin and I came up with two challenges we know for a fact those servicing clients in this area will face, as well as some words to the wise on how to address them head on.
One app doesn’t fit all
Here’s the number one biggest challenge we see, in development, today: A client will tell us they would like an app. They will say it needs to be “mobile friendly” and look good on iPad. They want some social elements, and it should work reasonably well in all common browsers, including IE, because that’s what they themselves (or their clients) use. We nod, as that sounds like an absolutely reasonable request, and we take on the project. Then the client sends over exactly one set of PSDs, with the all of the elements fit tightly together like Tetris blocks, in one painstakingly crafted presentation the size and shape of just one device. Kevin promptly cringes.
You see, it is very, very difficult for many of us in this industry to scale our visualization skills out exponentially, and design not just one user experience, but all of the user experiences implied by such a request. The window to your app might be tall, it might be wide, it might be huge, and it might be tiny. It might be served via FiOS, it might be served over a mobile connection with a limited data plan. The user might be navigating with their finger on a touch screen, or maybe with a mouse. How will swipe gestures work on devices where there is no such thing as swiping? Not daunted yet? Let’s talk Retina displays. Yeah, they’re on laptops now. Oh, and can it be a Facebook app? Wait, how many different versions of Android are there? Stop the madness.
OK, OK, I’ll stop. The point here is that one app doesn’t fit all devices. To stay wise — remember that and convey it to your agency partners and clients who ask you for this version of impossible. The industry is just not there yet. Maybe after a few years of hammering these challenges out, we’ll get one hot summer where it’s all pretty easy, before we’re all putting chips into our brains that will give us our email and entertainment, which will simply implant all relevant data directly into our neurons. I can’t wait to see how those apps will be QA’d. If you are pitching new business or bringing in clients to your shop, you must manage expectations in this area, or you will find yourself in over your head quickly.
Identify and spend time with your targets
The best thing you can do for yourself, as a brand or agency, is to pick your battles wisely, and respect the fact that each of them may have to be waged differently. It’s by no means impossible to build a responsive web application that can flex from a tall iPhone screen all the way out to a 1440×900 HD display, but you’re really going to have to think about every movement. Not just of your content, but also of the user: How will they navigate with their finger? How will they navigate with a mouse? How will you keep their phone from having to download too much, while maintaining a rich experience on their laptop over WiFi? Solving these challenges are what we’re built for, but we all have to do a better job of identifying them and being very precise about requirements.
Your client wants a social element. Stop, and just ask yourself (and then them) “why.” Is it because you want to be able to say that your app was social? Or is it because you want your app to augment a genuinely social experience that gets people talking about your brand? Yes, I know, the former, but the latter isn’t all that hard, if you want some extra credit.
So, how do you get wise and really drill into your target? You do this by being real. Picture real, actual people using your app. Think of someone you know in (or near) your target demographic. Talk to them, if you haven’t in awhile. I don’t need to tell you to look at the hit viral campaigns that blow up your Facebook feed, but if you’re going to copy them (no judgment), you must understand that it’s usually the variables, and rarely the formula that resonates with people. When a campaign is really successful, there’s always that “x factor,” that something that touched, amused, or impressed a large percentage of all that caught a glance of it. So if you’re going to steal, don’t steal the successful campaign’s formula. Go find someone inspiring, funny, or super smart. Steal their ideas, and then try plugging those into the formula that worked.
Kevin Cogill is CTO of Rey Interactive.
On Twitter? Follow iMedia Connection at @iMediaTweet.
“Businessman sitting at a desk” image via Shutterstock.
Qualtrics Site Intercept – A Real-Time Intelligence Service From A Family Run Business With $70 Million In Funding
Qualtrics has released its Site Intercept technology. It’s a hallmark event for the family-run Provo, Utah company. Now it’s time to see if it really will disrupt the stodgy world of market research and change how we view the ways data gets collected and acted upon in real-time for online marketing purposes.
Accel Partners and Sequoia Capital have a $70 million bet that the company can do it. But it’s still a question mark. The market research world is one that does a lot of business selling phone-based and other traditional survey techniques. The projects get expensive, costing hundreds of thousands of dollars over the span of a year.
There are signs that the market is ready to pop. The buzz about big data certainly helps as it is fueling a deeper interest in data analytics. A DIY ethos is also spreading in the enterprise. And the more modern forms of market research technology are also being used for more general purposes such as lead generation.
Site Intercept is positioned well in this converging market. The SaaS based offering, previously available through invite only, allows any marketer the capability to build their own market research and real-time marketing engine. Site Intercept is like a thin app that can be slid anywhere in a web site to get immediate feedback. It can be programmed for the time of day, geo-location, visit history, shopping cart content, etc. It lets brands display custom messages, surveys, promotions and other content to get immediate feedback. The interface is entirely point-and-click and drag-and-drop. It includes nine types of creative intercepts that marketers can use to interact with their customers such as pop-overs, pop-unders, customer embedded content and A/B testing.
The Qualtrics story is also a reminder of the almost old-fashioned concept of the family run business but with its own characteristics of the modern startup.
Qualtrics is owned and managed by the Smith family. There is Scott, the dad, an academic and pioneer in the field of market research who began experimenting with online surveys back in the 1990s. Ryan is the CEO and co-founder, who soon joined his dad to sell the hosted software developed by a group of graduate students. And then there is Jared, also a co-founder, who spent six years at Google scaling operations and leading all of the company’s product efforts for Greater China and Southeast Asia before moving on to Qualtrics full-time.
Qualtrics has more than 200 people in its Provo office. Its technology is used by a host of brands including Ebay, GM and Microsoft. The company competes with the likes of online survey companies like Survey Monkey and Checkbox.
But to really break through, it will have to become far more than market research technology. It will need to be viewed as a key method for capturing leads, personalizing content and getting that all important intelligence from the big data trove the market increasingly wants to explore.
(Smith family image via Forbes)
Anyone who follows my blog knows that I’m not a big fan of Klout, or any service that oversimplifies the complex process of assessing online influence by boiling it down to a single number. However, I do think it’s important that organizations be able to understand the online influence of people they want to build relationships with.
Awareness Networks just announced a tool that takes an intelligent and customized approach to influence assessment. The Social Marketing Automation suite enables customers to identify patterns in public online conversations, extract profile information and create what amounts to custom Klout scores.
Here’s how it might work: A user could search Twitter for people who have engaged directly with a brand more than twice over the last month, have mentioned the brand more than five times and have more than a specified number of followers. The suite can also dig into publicly available profile information to add filters by location, profession or any other data that is publicly available on Facebook or Twitter. So if you’re looking for health care professionals in the Milwaukee area who frequently recommend Motrin over Advil, you can find them for prospecting or a targeted marketing campaign.
Awareness goes a step further by combining public profile data with conversation topics to create prospect databases. This information can be imported into CRM and marketing automation packages, easing what is usually a laborious manual process. Integration with Salesforce.com is built into the first product and most of the leading platforms will be added over time, according to Mike Lewis, VP of marketing at Awareness. This addresses the problem of lead quality, which is the biggest cause of sales waste.
Awareness doesn’t extract data from social networks directly but rather works with Gnip, a company that has license agreements with most of the top social networks to distribute their content. About the only major source Gnip doesn’t have is LinkedIn, which keeps its profile information close to the vest. But YouTube, Tumblr, WordPress and many other sources are pumped through its firehose.
Competitive advantage is fleeting in this business, and I expect that others will quickly add this kind of functionality. Awareness’ strategy is smart: It will focus on providing the core data mining and filtering technology and work with partners to deliver results to whatever marketing or sales automation tool they prefer. Victory will go to the swiftest.
Pricing hasn’t been announced yet, but there’s a webinar set for Tuesday, Aug. 14 at 2 EDT at which more details will be discussed. Maybe you can pry some dollar figures out of the speakers then.
Full disclosure: I have been a paid consultant to Awareness on spot projects in the past, although I’ve done no work for the company in at least two years.