Skip to main content

TR Memescape

  • Talk Rational: We take Nothing for Granted.

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Testy Calibrate

Sports / Re: Winter Olympics 2018
Bold. Very bold. Putin must think he has an ace card in cyber that can bring down any opponent if necessary or something.
Statistics came well before computers. It would be very different if it were the other way around.

The stats most people learn in high school or college come from the time when computations were done with pen and paper. "Statistics were constrained by the computational technology available at the time," says Stanford statistics professor Robert Tibshirani. "People use certain methods because that is how it all started and that's what they are used to. It's hard to change it."

People who have taken intro statistics courses might recognize terms like "normal distribution," "t-distribution," and "least squares regression." We learn about them, in large part, because these were convenient things to calculate with the tools available in the early 20th century. We shouldn't be learning this stuff anymore--or, at least, it shouldn't be the first thing we learn. There are better options.

As a former data scientist, there is no question I get asked more than, "What is the best way to learn statistics?" I always give the same answer: Read An Introduction to Statistical Learning. Then, if you finish that and want more, read The Elements of Statistical Learning. These two books, written by statistics professors at Stanford University, the University of Washington, and the University Southern California, are the most intuitive and relevant books I've found on how to do statistics with modern technology. Tibsharani is a coauthor of both. You can download them for free.
Number crunchers

The books are based on the concept of "statistical learning," a mashup of stats and machine learning. The field of machine learning is all about feeding huge amounts of data into algorithms to make accurate predictions. Statistics is concerned with predictions as well, says Tibshirani, but also with determining how confident we can be about the importance of certain inputs.

This is important in areas like medicine, where a researcher doesn't just want to know whether a medicine worked, but also why it worked. Statistical learning is meant to take the best ideas from machine learning and computer science, and explain how they can be used and interpreted through a statistician's lens.

The beauty of these books is that they make seemingly impenetrable concepts--"cross-validation," "logistical regression," "support vector machines"--easily understandable. This is because the authors focus on intuition rather than mathematics. Unlike many statisticians, Tibshirani and his coauthors don't come from a math background. He believes this helps them think conceptually. "We try to explain [concepts] intuitively by explaining the underlying idea first," he says. "Then we give examples of a situation you would expect it work. And also, a situation where it might not work. I think people really appreciate that." I certainly did.

For example, a section of An Introduction to Statistical Learning is dedicated to explaining the use of "bootstrapping"--a statistical technique only available in the age of computers. Bootstrapping is a way to assess the accuracy of an estimate by generating multiple datasets from the same data.

For example, lets say you collected the weights of 1,000 randomly selected adult women in the US, and found that the average was 130 pounds. How confident can you be in this number? In conventional statistics, to answer this question you would use a formula developed more than a century ago, which relies on many assumptions. Today, rather than make those assumptions, you can use a computer to take thousands of samples of 500 people from your original 1,000 (this is the bootstrapping) and see how many of these results are close to 130. If most of them are, you can be more confident in the estimate.
Theory and application

These books, mercifully, don't require high-level math, like multivariate calculus or linear algebra. (If you're into that sort of thing, there is a wealth of worthy but dry academic literature out there for you.) "While knowledge of those topics is very valuable, we believe that they are not required in order to develop a solid conceptual understanding of how statistical learning methods work, and how they should be applied," says Daniela Witten, a coauthor of An Introduction to Statistical Learning.

Helpfully, the books also provide code you can use to apply the tools with the statistical programming language R. I recommend putting their examples to work on a dataset you are excited about. If you are into novels, use it to analyze Goodreads ratings. If you like basketball, apply their examples to numbers at Basketball Reference. The statistical learning tools are wonderful in themselves, but I've found they work best for people who are motivated by a personal or professional project.

Data and statistics are an increasingly important part of modern life, and nearly everyone would be better off with a deeper understanding of the tools that help explain our world. Even if you don't want to become a data analyst--which happens to be one of the fastest-growing jobs out there, just so you know--these books are invaluable guides to help explain what's going on.
lots of the text is links at the article.
heh. yes.
and this:
To wit, the tweets said that the online advertising campaign led by the shadowy Internet Research Agency was meant to divide the American people, not influence the 2016 election.

Antonio García Martínez (@antoniogm) is an Ideas contributor for WIRED. Before turning to writing, he dropped out of a doctoral program in physics to work on Goldman Sachs' credit trading desk, then joined the Silicon Valley startup world, where he founded his own startup (acquired by Twitter in 2011) and finally joined Facebook's early monetization team, where he headed the company's targeting efforts. His 2016 memoir, Chaos Monkeys, was a New York Times best seller and NPR Best Book of the Year, and his writing has appeared in Vanity Fair, The Guardian, and The Washington Post. He splits his time between a sailboat on the San Francisco Bay and a yurt in Washington's San Juan Islands.

You're probably skeptical of Rob's claim, and I don't blame you. The world looks very different to people outside the belly of Facebook's monetization beast. But when you're on the inside, like Rob is and like I was, and you have access to the revenue dashboards detailing every ring of the cash register, your worldview tends to follow what advertising data can and cannot tell you.

From this worldview, it's still not clear how much influence the IRA had with its Facebook ads (which, as others have pointed out, is just one small part of the huge propaganda campaign that Mueller is currently investigating). But no matter how you look at them, Russia's Facebook ads were almost certainly less consequential than the Trump campaign's mastery of two critical parts of the Facebook advertising infrastructure: The ads auction, and a benign-sounding but actually Orwellian product called Custom Audiences (and its diabolical little brother, Lookalike Audiences). Both of which sound incredibly dull, until you realize that the fate of our 242-year-old experiment in democracy once depended on them, and surely will again.

Like many things at Facebook, the ads auction is a version of something Google built first. As on Google, Facebook has a piece of ad real estate that it's auctioning off, and potential advertisers submit a piece of ad creative, a targeting spec for their ideal user, and a bid for what they're willing to pay to obtain a desired response (such as a click, a like, or a comment). Rather than simply reward that ad position to the highest bidder, though, Facebook uses a complex model that considers both the dollar value of each bid as well as how good a piece of clickbait (or view-bait, or comment-bait) the corresponding ad is. If Facebook's model thinks your ad is 10 times more likely to engage a user than another company's ad, then your effective bid at auction is considered 10 times higher than a company willing to pay the same dollar amount.

A canny marketer with really engaging (or outraging) content can goose their effective purchasing power at the ads auction, piggybacking on Facebook's estimation of their clickbaitiness to win many more auctions (for the same or less money) than an unengaging competitor. That's why, if you've noticed a News Feed ad that's pulling out all the stops (via provocative stock photography or other gimcrackery) to get you to click on it, it's partly because the advertiser is aiming to pump up their engagement levels and increase their exposure, all without paying any more money.

During the run-up to the election, the Trump and Clinton campaigns bid ruthlessly for the same online real estate in front of the same swing-state voters. But because Trump used provocative content to stoke social media buzz, and he was better able to drive likes, comments, and shares than Clinton, his bids received a boost from Facebook's click model, effectively winning him more media for less money. In essence, Clinton was paying Manhattan prices for the square footage on your smartphone's screen, while Trump was paying Detroit prices. Facebook users in swing states who felt Trump had taken over their news feeds may not have been hallucinating.

(Speaking of Manhattan vs. Detroit prices, there are some (very nonmetaphorical) differences in media costs across the country that also impacted Trump's ability to reach voters. Broadly, advertising costs in rural, out-of-the-way areas are considerably less than in hotly contested, dense urban areas. As each campaign tried to mobilize its base, largely rural Trump voters were probably cheaper to reach than Clinton's urban voters. Consider Germantown, Pa. (a Philly suburb Clinton won by a landslide) vs. Belmont County, Ohio (a rural county Trump comfortably won). Actual media costs are closely guarded secrets, but Facebook's own advertiser tools can give us some ballpark estimates. For zip code 43950 (covering the county seat of St. Clairsville, Ohio), Facebook estimates an advertiser can show an ad to about 83 people per dollar. For zip code 19144 in the Philly suburbs, that number sinks to 50 people an ad for every dollar of ad spend. Averaged over lots of time and space, the impacts on media budgets can be sizable. Anyway ...)

    The Like button is our new ballot box, and democracy has been transformed into an algorithmic popularity contest.

The above auction analysis is even more true for News Feed, which is only based on engagement, with every user mired in a self-reinforcing loop of engagement, followed by optimized content, followed by more revealing engagement, then more content, ad infinitum. The candidate who can trigger that feedback loop ultimately wins. The Like button is our new ballot box, and democracy has been transformed into an algorithmic popularity contest.

But how to trigger the loop? For that, we need the machinery of targeting. (Full disclosure: I was the original product manager for Custom Audiences, and along with a team of other product managers and engineers, I launched the first versions of Facebook precision targeting in the summer of 2012, in those heady and desperate days of the IPO and sudden investor expectation.)

Despite folklore about "selling your data," most Facebook advertisers couldn't care less about your Likes, your drunk college photos, or your gossipy chats with a boyfriend. What advertisers want to do is find the person who left a product unpurchased in an online shopping cart, just used a loyalty card to buy diapers at Safeway, or registered as a Republican voter in Stark County, Ohio (a swing county in a swing state).

Custom Audiences lets them do that. It's the tunnel beneath the data wall that allows the outside world into Facebook's well-protected garden, and it's like that by design.

Browsed for shoes and then saw them on Facebook? You're in a Custom Audience.

Registered for an email newsletter or used your email as login somewhere? You're in a Custom Audience.

Ordered something to a postal address known to merchants and marketers? You're definitely in a Custom Audience.

Here's how it works in practice:

A campaign manager takes a list of emails or other personal data for people they think will be susceptible to a certain type of messaging (e.g. people in Florida who donated money to Trump For America). They upload that spreadsheet to Facebook via the Ads Manager tool, and Facebook scours its user data, looks for users who match the uploaded spreadsheet, and turns the matches into an "Audience," which is really just a set of Facebook users.

Facebook can also populate an audience by reading a user's cookies--those digital fragments gathered through a user's wanderings around the web. Half the bizarre conspiracy theories around Facebook targeting boil down to you leaving a data trail somewhere inside our consumer economy that was then uploaded via Custom Audiences. In the language of database people, there's now a "join" between the Facebook user ID (that's you) and this outside third-party who knows what you bought, browsed, or who you voted for (probably). That join is permanent, irrevocable, and will follow you to every screen where you've used Facebook.

The above is pretty rudimentary data plumbing. But only when you've built a Custom Audience can you build Lookalike Audiences-- the most unknown, poorly understood, and yet powerful weapon in the Facebook ads arsenal.

With a mere mouse click from our hypothetical campaign manager, Facebook now searches the friends of everyone in the Custom Audience, trying to find everyone who (wait for it) "looks like" you. Using a witches' brew of mutual engagement--probably including some mix of shared page Likes, interacting with similar News Feed or Ads content, a score used to measure your social proximity to friends--the Custom Audience is expanded to a bigger set of like-minded people. Lookalikes.

(Another way to picture it: Your social network resembles a nutrient-rich petri dish, just sitting out in the open. Custom Audiences helps mercenary marketers find that dish, and lets them plant the bacterium of a Facebook post inside it. From there, your own interaction with the meme, which is echoed in News Feed, spreads it to your immediate vicinity. Lookalike Audiences finishes the job by pushing it to the edges of your social petri dish, to everyone whose tastes and behaviors resemble yours. The net result is a network overrun by an infectious meme, dutifully placed there by an advertiser, and spread by the ads and News Feed machinery.)

read more at the link
As Zeynep Tufekci has eloquently argued: "The most effective forms of censorship today involve meddling with trust and attention, not muzzling speech itself."

So we also get subjected to all this intentional padding, applied selectively, to defuse debate and derail clear lines of argument; to encourage confusion and apathy; to shift blame and buy time. Bored people are less likely to call their political representatives to complain.

Truly fake news is the inception layer cake that never stops being baked. Because pouring FUD onto an already polarized debate -- and seeking to shift what are by nature shifty sands (after all information, misinformation and disinformation can be relative concepts, depending on your personal perspective/prejudices) -- makes it hard for any outsider to nail this gelatinous fakery to the wall.

Why would social media platforms want to participate in this FUDing? Because it's in their business interests not to be identified as the primary conduit for democracy damaging disinformation.

And because they're terrified of being regulated on account of the content they serve. They absolutely do not want to be treated as the digital equivalents to traditional media outlets.

But the stakes are high indeed when democracy and the rule of law are on the line. And by failing to be pro-active about the existential threat posed by digitally accelerated disinformation, social media platforms have unwittingly made the case for external regulation of their global information-shaping and distribution platforms louder and more compelling than ever.



very gun outrage in America is now routinely followed by a flood of Russian-linked Twitter bot activity. Exacerbating social division is the name of this game. And it's playing out all over social media continually, not just around elections.

In the case of Russian digital meddling connected to the UK's 2016 Brexit referendum, which we now know for sure existed -- still without having all of the data we need to quantify the actual impact, the chairman of a UK parliamentary committee that's running an enquiry into fake news has accused both Twitter and Facebook of essentially ignoring requests for data and help, and doing none of the work the committee asked of them.

Facebook has since said it will take a more thorough look through its archives. And Twitter has drip-fed some tidbits of additional infomation. But more than a year and a half after the vote itself, many, many questions remain.

And just this week another third party study suggested that the impact of Russian Brexit trolling was far larger than has been so far conceded by the two social media firms.

The PR company that carried out this research included in its report a long list of outstanding questions for Facebook and Twitter.

Here they are:

    How much did [Russian-backed media outlets] RT, Sputnik and Ruptly spend on advertising on your platforms in the six months before the referendum in 2016?
    How much have these media platforms spent to build their social followings?
    Sputnik has no active Facebook page, but has a significant number of Facebook shares for anti-EU content, does Sputnik have an active Facebook advertising account?
    Will Facebook and Twitter check the dissemination of content from these sites to check they are not using bots to push their content?
    Did either RT, Sputnik or Ruptly use 'dark posts' on either Facebook or Twitter to push their content during the EU referendum, or have they used 'dark posts' to build their extensive social media following?
    What processes do Facebook or Twitter have in place when accepting advertising from media outlets or state owned corporations from autocratic or authoritarian countries? Noting that Twitter no longer takes advertising from either RT or Sputnik.
    Did any representatives of Facebook or Twitter pro-actively engage with RT or Sputnik to sell inventory, products or services on the two platforms in the period before 23 June 2016?

We put these questions to Facebook and Twitter.

In response, a Twitter spokeswoman pointed us to some "key points" from a previous letter it sent to the DCMS committee (emphasis hers):

    In response to the Commission's request for information concerning Russian-funded campaign activity conducted during the regulated period for the June 2016 EU Referendum (15 April to 23 June 2016), Twitter reviewed referendum-related advertising on our platform during the relevant time period.

    Among the accounts that we have previously identified as likely funded from Russian sources, we have thus far identified one account--@RT_com-- which promoted referendum-related content during the regulated period. $1,031.99 was spent on six referendum-related ads during the regulated period. 

    With regard to future activity by Russian-funded accounts, on 26 October 2017, Twitter announced that it would no longer accept advertisements from RT and Sputnik and will donate the $1.9 million that RT had spent globally on advertising on Twitter to academic research into elections and civil engagement. That decision was based on a retrospective review that we initiated in the aftermath of the 2016 U.S. Presidential Elections and following the U.S. intelligence community's conclusion that both RT and Sputnik have attempted to interfere with the election on behalf of the Russian government. Accordingly, @RT_com will not be eligible to use Twitter's promoted products in the future.

The Twitter spokeswoman declined to provide any new on-the-record information in response to the specific questions.

A Facebook representative first asked to see the full study, which we sent, then failed to provide a response to the questions at all.

The PR firm behind the research, 89up, makes this particular study fairly easy for them to ignore. It's a pro-Remain organization. The research was not undertaken by a group of impartial university academics. The study isn't peer reviewed, and so on.

But, in an illustrative twist, if you Google "89up Brexit", Google New injects fresh Kremlin-backed opinions into the search results it delivers -- see the top and third result here...

worth reading the whole thing
As it turns out she does believe that errors under my definition are necessary to ensure long lineages. But she's wrong. She, like many evolutionists, believe this on faith alone because there is absolutely no evidence for this whatsoever. In fact the whole Third Way movement got started because many scientists are finally beginning to realize this.
are you trying to convince yourself?
I have heard him on the radio but I don't know if I've ever seen him on tv. Plus I've read a few of his columns. He seems like a capitalist of the moderately Hayek variety. That kinda makes his whole take on everything flawed in my opinion so I don't really pay much attention to him. Should I? Does he ever make intelligent points or are they basically cheerleading?
Yeah, sure, maybe he can do without it. But he can't win the specific battle over Trump needing to testify, if forcing him takes years.
If he's charged with a crime, he can be forced to testify. No?
Did you miss a "not"?
yeah maybe. And as a defendant you can't be forced to testify anyway I guess.
Yeah, sure, maybe he can do without it. But he can't win the specific battle over Trump needing to testify, if forcing him takes years.
If he's charged with a crime, he can be forced to testify. No?
Unless Mueller could get enough evidence for an indictment without his testimony. Right?
natesilver: Again, a lot of this is just that David Brooks had a party (the GWB-era GOP) that he once mostly agreed with and now he doesn't have one. Which is annoying for David Brooks but doesn't really provide much evidence either way in terms of broader public sentiment.
President Donald Trump made tens of millions of dollars in profits by allowing Colombian drug cartels and other groups to launder money through a Trump-affiliated hotel in Panama, according to a new investigation by the organization Global Witness.

In the early 2000s, Trump was having financial difficulties and began selling his high-profile name to real estate developers around the world, the report said. One of these developed Panama's Trump Ocean Club International Hotel and Tower.

The report said the drug cartels purchased hotel units to hide the origins of money earned through drug trafficking and other criminal activity, and Trump is estimated to have earned tens of millions of dollars from the deals.

Keep up with this story and more by subscribing now

Some observers are saying it is time for Congress to begin investigating the president's finances and potential conflicts of interest.

"This is inherently a political problem," Alex Howard, deputy director of the Sunlight Foundation, told Newsweek. "The government can investigate a company, even the president's company. The problem here is that it's about the president, and Congress is not holding him accountable for what he has done in this context. They aren't holding hearings about the Trump Organization, and the president himself is not being transparent."

The Sunlight Foundation has compiled a list of what it claims are Trump's conflicts of interest: It numbers more than 600 for the president and 1,100 for the first family. 

The report said the Panama project is a textbook case of money laundering.

"Investing in luxury properties is a tried and trusted way for criminals to move tainted cash into the legitimate financial system, where they can spend it freely," the report noted. "Once scrubbed clean in this way, vast profits from criminal activities like trafficking people and drugs, organized crime, and terrorism can find their way into the U.S. and elsewhere."

"In the case of the Trump Ocean Club, accepting easy - and possibly dirty - money early on would have been in Trump's interest; a certain volume of pre-construction sales was necessary to secure financing for the project, which stood to net him $75.4 million by the end of 2010."

Numerous investigations have shown that Trump rarely looks into the people he hires or does business with. Instead, observers say he has a pattern of entering into business deals with people suspected of money laundering and corruption. The business in Panama was an example.

One of the men involved in the scheme was David Eduardo Helmut Murcia Guzmán, who a U.S. court subsequently sentenced to nine years in prison for laundering millions of dollars. Another was Alexandre Henrique Ventura Nogueira, who sold units at the Trump Ocean Club and later admitted that some of the people he did business with were members of the Russian mafia.

Trump family members were allegedly involved in directly managing the Panama project.

The White House did not immediately respond to request for comment.
It's like quoting from hitler to make the argument that nazis weren't concerned about jews.
... But the point is that without some imperfection in the reproduction process, i.e. without most, or at least some, offspring being unique, populations are doomed not to adapt. Which is why small populations tend to be vulnerable to extinction. Which is also why the Ark story is so bloody silly, but I do realise that the subtext here is that somehow you've got to get a lot of extra genetic variance into those animal pairs but you can't bring yourself to call them "errors".  But you won't get them from recombination either.  You need new alleles, which means that at least some of the gene sequences need to split and recombine mid-gene in a manner that will produce a gene different from both parents.  Some would call this an "error" in the recombination process.
This was the crux of the issue back in the "Who says Adam didn't have HUNDREDS of alleles?"  days, and Hawkins seems to have made no progress on it since.

Where DID the tens / hundreds / thousands of alleles per locus in animal genomes come from ?

Hint: Spoiler (click to show/hide)

I'm getting off Pingu's latest stupid merry go-round ...

But this from Voxrat is interesting ...

I will revisit it
What an apt choice of words.
OK listen ...

Let's adopt a definition of "error" that we can both use ...

How about this ... "a difference from the parent sequence that escapes the error correction mechanism because of an error correction system malfunction" ... this would not include recombination or HGT.

Can we agree on this definition?

If so, then I'd like to know if you stand by your statement 
Therefore populations in which the "error" rate is low but non-zero are likely to adapt best and thus leave long lineages.

Did you hear about the mansplainer who fell down a well?

Actually, it was a manhole.
You do NOT need a "non-zero error rate" to ensure long lineages in bacteria because of HGT.
You do not need HGT to ensure long lineages in bacteria.
One of your fundamental
This is your basic misunderstanding.
Your basic misunderstanding is thinking that you understand this stuff.

See: Dunning-Kruger effect.

Your misunderstanding is that because I don't know everything, I must not know anything.

I obviously know more about this one narrow topic than Pingu.
It's hard to believe you know anything. But pingu (and everyone else) can see clearly that you don't understand the topic well enough to learn more about it.
I actually think I persisted in that error for only a few hours at which time I corrected it. And that was 12 years ago. And yes I know a lot more now. Apparently I know a lot more than Pingu.
Heh. The weirdness of that idea could generate several PhDs on human cognition.
By realizing that, by the definition she used, either recombination is an "error", or it doesn't even make sense to ask if it is an error.
YOU are the one not reading carefully. 

When she talks about an "error rate" that's low but non-zero being best for adaptability and long lineages, she is definitely not referring to regular old vanilla recombination.  She's talking about the same thing the Nature article describes which the cell works very hard to prevent, but it happens an little bit anyway.

You people are idiots.
She explicitly said "i.e. in which every offspring is identical to its parent"

That doesn't happen with recombination.
I assumed she meant "perfect recombination" ... which living cells approach closely, but don't quite ever achieve.

If she didn't mean that, then WTH did she mean?
Your stupidity is all encompassing
I cannot see how anyone can explain away the fact that this IS what she thinks.

And it's incorrect.

Good God you are an idiot Dave
Lucky you. Just change your last name to Johnson and badabing you belong!
There's no such thing as an error ... or a mistake.  So I'm told.
you got a lot of participation awards, eh?
So Dave,

Still looking for that "100% truth"?
I would think, like Dave, they would just ignore that sort of thing.
B-b-bbut, the rwnj's have such good insult names for adam schiff. He MUST BE LYING!