Citation Formats on Crossref

Real-time Stream of DOIs being cited in Wikipedia

Joe Wass — Tue, 03 Mar 2015 00:00:00 +0000

TL;DR

Watch a real-time stream of DOIs being cited (and “un-cited!” ) in Wikipedia articles across the world: https://live-eventdata-crossref-org.pluma.sjfc.edu/live.html

Background

For years we’ve known that the Wikipedia was a major referrer of Crossref DOIs and about a year ago we confirmed that, in fact, the Wikipedia is the 8th largest refer of Crossref DOIs. We know that people follow the DOIs, too. This despite a fraction of Wikipedia citations to the scholarly literature even using DOIs. So back in August we decided to create a Wikimedia Ambassador programme. The goal of the programme was to promote the use of persistent identifiers in citation and attribution in Wikipedia articles. We would do this through outreach and through the development of better citation-related tools.

Remember when we originally wrote about our experiments with the PLOS ALM code and how that has transitioned into the DOI Event Tracking Pilot? In those posts we mentioned that one of the hurdles in gathering information about DOI events is the actual process of polling third party APIs for activity related to millions of DOIs. Most parties simply wouldn’t be willing handle the load of a 100K API calls an hour. Besides, polling is a tremendously inefficient process, only a fraction of DOIs are ever going to generate events, but we’d have to poll for each of them, repeatedly, forever, to get an accurate picture of DOI activity. We needed a better way. We needed to see if we could reverse this process and convince some parties to instead “push” us information whenever they saw DOI related events (e.g. citations, downloads, shares, etc). If only we could convince somebody to try this…

Wikipedia DOI Events

In December 2014 we took the opportunity of the 2014 PLOS/Crossref ALM Workshop in San Francisco too meet with Max Klein and Anthony Di Franco where we kicked off a very exciting project.

There’s always someone editing a Wikipedia somewhere in the world. In fact, you can see a dizzying live stream of edits. We thought that given that there are so many DOIs in Wikipedia, that live stream may contain some diamonds (DOIs are made of diamond, that’s how they can be persistent). Max and Anthony went away and came back with a demo that contains a surprising amount of DOI activity.

That demo is evolving into a concrete service, called Cocytus. It is running at Wikimedia Labs monitoring live edits as you read this.

For now we’re feeding that data into the DOI Events Collection app (which is an off-shoot of the Chronograph project). We are in the process of modifying the Lagotto code so that we can instead push those events into the DOI Event Tracking Instance.

The first DOI event we noticed was delightfully prosaic: The DOI for “The polymath project” is cited by the Wikipedia page for “Polymath Project”. Prosaic perhaps, but the authors of that paper probably want to know. Maybe they can help edit the page.

Or how about this. Someone wrote a a paper about why people edit Wikipedia and then it was cited by Wikipedia. And then the citation was removed. The plot thickens…

We’re interested in seeing how DOIs are used outside of the formal scholarly literature. What does that mean? We don’t fully know, that’s the point. We have retractions in scholarly literature (and our Crossmark metadata and service allow publishers to record that), but it’s a bit different on Wikipedia. Edit wars are fought over … well you can see for yourself.

Citations can slip in and out of articles. We saw the DOI 10.1001/archpediatrics.2011.832 deleted from “Bipolar disorder in children”. If we’d not been monitoring the live feed (we had considered analysing snapshots of the Wikipedia in bulk) we might never have seen that. This is part of what non-traditional citations means, and it wasn’t obvious until we’d seen it.

You can see this activity on the Chronograph’s stream. Or check your favourite DOI. Please be aware that we’re only collecting newly added citations as of today. We do intend to go back and back-fill, but that may take some time- as it * cough * requires polling again.

Some Technical Things

A few interesting things that happened as a result of all this:

Secure URLs

SSL and HTTPS were invented so you could do things like banking on the web without fear of interception or tampering. As the web becomes a more important part of life, many sites are upgrading from HTTP to HTTPS, the secure version. This is not only because your confidential details may be tampered with, but because certain governments might not like you reading certain materials.

Because of this, some time ago, Wikipedia decided to embark on an upgrade to HTTPS last year, and they are a certain way along the path. The IDF, who are responsible for running the DOI system, upgraded to HTTPS this Summer, although most DOIs are referred to by HTTP still.

We met with Dario Taraborelli at the ALM workshop and discussed the DOI referral data that is fed into the Chronograph. We put two and two together and realised that Wikipedia was linking to DOIs (which are mostly HTTP) from pages which might be served over HTTPS. New policies in HTML5 specify that referrer URL headers shouldn’t be sent from HTTPS to HTTP (in case there was something secret in them). The upshot of this is, if someone’s browsing Wikipedia via HTTPS and click on a normal DOI, we won’t know that the user came from Wikipedia. Not a huge problem today, but as Wikipedia switches over to entirely secure, we’re going to miss out on very useful information.

Fortunately, the HTML5 specification includes a way to fix this (without leaking sensitive information). We discussed this with Dario, and he did some research, and came up with a suggestion, which got discussed. It’s fascinating to watch a democratic process like this take place and take part in it.

We’re waiting to see how the discussion turns out, and hope that it all works out so we can continue to report on how amazing Wikipedia is at sending people to scholarly literature.

How shall I cite thee?

Another discussion grew out of that process, and we started talking to a Wikipedian called Nemo (note to Latin scholars: we weren’t just talking to ourselves). Nemo (real name Federico Leva) had a few suggestions of his own. Another way to solve the referrer problem is by using HTTPS URLs (HTML5 allows browsers to send the referrer domain when going from HTTPS to HTTPS).

This means going back to all the articles that use DOIs and change them from HTTP to HTTPS. Not as simple as it sounds, and it doesn’t sound simple. We started looking into how DOIs were cited on Wikipedia.

After some research we found that there are more ways that we expected to cite DOIs.

First, there’s the URL. You can see it in action in this article. URLs can take various forms.

Second there’s the official template tag, seen in action here:

<ref name="SCI-20140731">{{cite journal |title=Sustained miniaturization and anatomical innovation in the dinosaurian ancestors of birds |url=http://www.sciencemag.org.pluma.sjfc.edu/content/345/6196/562 |date=1 August 2014 |journal=[[Science (journal)|Science]] |volume=345 |issue=6196 |pages=562–566 |doi=10.1126/science.1252243 |accessdate=2 August 2014 |last1=Lee |first1=Michael S. Y. |first2=Andrea|last2=Cau |first3=Darren|last3=Naish|first4=Gareth J.|last4=Dyke}}</ref>

There’s a DOI in there somewhere. This is the best way to cite DOIs, firstly as it’s actually a proper traditional citation and there’s nothing magic about DOIs, secondly because it’s a template tag and can be re-rendered to look slightly different if needed.

Third there’s the old official DOI template tag that’s now discouraged:

<ref name="Example2006">{{Cite doi|10.1146/annurev.earth.33.092203.122621}}</ref>

And then there’s another one.

{{doi|10.5555/123456789}}

Knowing all this helps us find DOIs. But if we want to convert DOIs links in Wikipedia to use HTTPS, it means that there are more template tags to modify and more pages to re-render.

Nemo also put DOIs on the Interwiki Map which should make automatically changing some of the URLs a lot easier.

We’re very grateful to Nemo for his suggestions and work on this. We’ll report back!

The elephant in the room

Those of you who know how DOIs work will have spotted an unsecured elephant in the room. When you visit a DOI, you visit the URL, which hits the DOI resolver proxy server, which returns a message to your browser to redirect to the landing page on the publisher’s site.

Securely talking to the DOI resolver by using HTTPS instead of HTTP means that no-one can eavesdrop and see which DOI you are visiting, or tamper with the result and send you off to a different page. But the page you are sent to will be, in nearly all cases, still HTTP. Upgrading infrastructure isn’t trivial, and, with over 4000 members (mostly publishers), most Crossref DOIs will still redirect to standard HTTP pages for the foreseeable future.

You can keep as secure as possible by using HTTPS Everywhere.

Fin

There’s lots going on, watch this space to see developments. Thanks for reading this, and all the links. We’d love to know what you think.

Bootnote

Not long after this blog post was published we saw something very interesting.

That’s no DOI. We like interesting things, but they can panic us. This turned out to be a great example of why this kind of thing can be useful. A minute’s digging and we found the article edit:

It turns out that this was a typo: someone put a title when they should have put in a DOI. And, as the event shows, this was removed from the Wikipedia article.

Crossref Metadata Search++

Geoffrey Bilder — Thu, 11 Oct 2012 00:00:00 +0000

We have just released a bunch of new functionality for Crossref Metadata Search. The tool now supports the following features:

A completely new UI
Faceted searches
Copying of search results as formatted citations using CSL
COinS, so that you can easily import results into Zotero and other document management tools
An API, so that you can integrate Crossref Metadata Search into your own applications, plugins, etc.
Basic OpenSearch support- so that you can integrate Crossref Metadata Search into your browser’s search bar.
Searching for a particular Crossref DOI
Searching for a particular Crossref ShortDOI
Searching for articles in a particular journal via the journal’s ISSN

At the moment, Crossref Metadata Search (CRMDS) is a Crossref Labs project and, as such, should be used with some trepidation. Our goal is to release CRMS as a production service ASAP, but we wanted to get public feedback on the service before making the move to a production system.

PDF-Extract

Geoffrey Bilder — Tue, 17 Apr 2012 00:00:00 +0000

PDF-EXTRACT

Crossref Labs is happy to announce the first public release of “pdf-extract” an open source set of tools and libraries for extracting citation references (and, eventually, other semantic metadata) from PDFs. We first demonstrated this tool to Crossref members at our annual meeting last year. See the pdf-extract labs page for a detailed introduction to this new set of tools.

If you are unable to download and install the tool, you can play with a experimental web interface called “Extracto.” Be warned, Extracto is running on very feeble server using an erratic and slow internet connection. The only guarantee that we can make about using it is that it will repeatedly fall over and annoy you. The weasel has spoken.

Turning DOIs into formatted citations

Karl Ward — Mon, 28 Nov 2011 00:00:00 +0000

Today two new record types were added to dx.doi.org resolution for Crossref DOIs. These allow anyone to retrieve DOI bibliographic metadata as formatted bibliographic entries. To perform the formatting we’re using the citation style language processor, citeproc-js which supports a shed load of citation styles and locales.

In fact, all the styles and locales found in the CSL repositories, including many common styles such as bibtex, apa, ieee, harvard, vancouver and chicago are supported. First off, if you’d like to try citation formatting without using content negotiation, there’s a simple web UI that allows input of a DOI, style and locale selection. If you’re more into accessing the web via your favorite programming language, have a look at these content negotiation curl examples. To make a request for the new “text/bibliography” record type: $ curl -LH “Accept: text/bibliography; style=bibtex” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 @article{Atkins_Gershell_2002, title={From the analyst’s couch: Selective anticancer drugs}, volume={1}, DOI={10.1038/nrd842}, number={7}, journal={Nature Reviews Drug Discovery}, author={Atkins, Joshua H. and Gershell, Leland J.}, year={2002}, month={Jul}, pages={491-492}} A locale can be specified with the “locale” record type parameter, like this: $ curl -LH “Accept: text/bibliography; style=mla; locale=fr-FR” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 Atkins, Joshua H., et Leland J. Gershell. « From the analyst’s couch: Selective anticancer drugs ». Nature Reviews Drug Discovery 1.7 (2002): 491-492. You may want to process metadata through CSL yourself. For this use case, there’s another new record type, “application/citeproc+json” that returns metadata in a citeproc-friendly JSON form: $ curl -LH “Accept: application/citeproc+json” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 {“volume”:“1”,“issue”:“7”,“DOI”:“10.1038/nrd842”,“title”:“From the analyst’s couch: Selective anticancer drugs”,“container-title”:“Nature Reviews Drug Discovery”,“issued”:{“date-parts”:[[2002,7]]},“author”:[{“family”:“Atkins”,“given”:“Joshua H.”},{“family”:“Gershell”,“given”:“Leland J.”}],“page”:“491-492”,“type”:“article-journal”} Finally, to retrieve lists of supported styles and locales, see:

* https://crosscite.org

style and locale repositories. There’s one big caveat to all this. The CSL processor will do its best with Crossref metadata which can unfortunately be quite patchy at times. There may be pieces of metadata missing, inaccurate metadata or even metadata items stored under the wrong field, all resulting in odd-looking formatted citations. Most of the time, though, it works.

Citation Typing Ontology

Geoffrey Bilder — Fri, 20 Mar 2009 00:00:00 +0000

I was happy to read David Shotton’s recent Learned Publishing article, Semantic Publishing: The Coming Revolution in scientific journal publishing, and see that he and his team have drafted a Citation Typing Ontology.^*

Anybody who has seen me speak at conferences knows that I often like to proselytize about the concept of the “typed link”, a notion that hypertext pioneer, Randy Trigg, discussed extensively in his 1983 Ph.D. thesis.. Basically, Trigg points out something that should be fairly obvious- a citation (i.e. “a link”) is not always a “vote” in favor of the thing being cited.
In fact, there are all sorts of reasons that an author might want to cite something. They might be elaborating on the item cited, they might be critiquing the item cited, they might even be trying to refute the item cited (For an exhaustive and entertaining survey of the use and abuse of citations in the humanities, Anthony Grafton‘s, The Footnote: A Curious History, is a rich source of examples)
Unfortunately, the naive assumption that a citation is tantamount to a vote of confidence has become inshrined in everything from the way in which we measure scholarly reputation, to the way in which we fund universities and the way in which search engines rank their results. The distorting affect of this assumption is profound. If nothing else, it leads to a perverse situation in which people will often discuss books, articles, and blog postings that they disagree with without actually citing the relevant content, just so that they can avoid inadvertently conferring “wuffie” on the item being discussed. This can’t be right.
Having said that, there has been a half-hearted attempt to introduce a gross level of link typology with the introduction of the “nofollow” link attribute- an initiative started by Google in order to try to address the increasing problem of “Spamdexing”. But this is a pretty ham-fisted form of link typing- particularly in the way it is implemented by the Wikipedia where Crossref DOI links to formally published scholarly literature have a “nofollow” attribute attached to them but, inexplicably, items with a PMID are not so hobbled (view the HTML source of this page, for example). Essentially, this means that, the Wikipedia is a black-hole of reputation. That is, it absorbs reputation (through links too the Wikipedia), but it doesn’t let reputation back out again. Hell, I feel dirty for even linking to it here ;-).
Anyway, scholarly publishers should certainly read Shotton’s article because it is full of good, and practical ideas about what can can be done with today’s technology in order to help us move beyond the “digital incunabula” that the industry is currently churning out. The sample semantic article that Shotton’s team created is inspirational and I particularly encourage people to look at the source file for the ontology-enhanced bibliography which reveals just how much more useful metadata can be associated with the humble citation.
And now I wonder whether CiteULike, Connotea, 2Collab or Zotero will consider adding support for the CItation Typing Ontology into their respective services?
* Disclosure:
a) I am on the editorial board of Learned Publishing
b) Crossref has consulted with David Shotton on the subject of semantically enhancing journal articles

Ubiquity commands for Crossref services

Geoffrey Bilder — Wed, 03 Dec 2008 00:00:00 +0000

So the other day Noel O’Boyle made me feel guilty when he pinged me and asked about the possibility using one of the Crossref APIs for creating a Ubiquity extension. You see, I had played with the idea myself and had not gotten around to doing much about it. This seemed inexcusable- particularly given how easy it is to build such extensions using the API we developed for the WordPress and Moveable Type plugins that we announced earlier in the year. So I dug up my half-finished code, cleaned it up a bit and have posted the results.

Note that the back-end that supports the plugins has been moved to more stable machines and the index is now being automatically updated with journal and conference proceeding deposits (sorry, no books yet).

Also note that we are hoping that others will look at the code for the WordPress, Moveable Type and Ubiquity plugins and create more such extensions. If you do, please let us know about them at citation-plugin@crossref.org.

Crossref Citation Plugin (for WordPress)

Crossref — Sat, 09 Feb 2008 00:00:00 +0000

OK, after a number of delays due to everything from indexing slowness to router problems, I’m happy to say that the first public beta of our WordPress citation plugin is available for download via SourceForge. A Movable Type version is in the works.

And congratulations to Trey at OpenHelix who became laudably impatient, found the SourceForge entry for the plugin back on February 8th and seems to have been testing it since. He has a nice description of how it works (along with screenshots), so I won’t repeat the effort here.

Having said that, I do include the text of the README after the jump. Please have a look at it before you install, because it might save you some mystification.

Description

A WordPress plugin that allows you to search Crossref metadata using citations or partial citations. When you find the reference that you want, insert the formatted and DOI-linked citation into your blog posting along with supporting COINs metadata. The plugin supports both a long citation format and a short (op. cit.) format.

Warnings, Caveats and Weasel Words

Please note the following about this plugin:

We are releasing this as a test. It is running on R&D equipment in a non-production environment and so it may disappear without warning or perform erratically. If it isn’t working for some reason, come back later and try again. If it seems to be broken for a prolonged period of time, then please report the problem to us via sourceforge.
There is currently a 20 item limit on the number of hits returned per query. This might seem arbitrary and stingy, but please remember- we are not trying to create a fully blown search engine- we’re just trying to create a citation lookup service. Of course, if, after looking at how the service is used, it looks like we need to up this limit, we will.
If you look in the plugin options (or at the code), you will see that the system includes an API key. At the moment we have no restrictions on use of this service, but have included this in case we need to protect the system from abuse.
The bulk of the functionality we have developed is actually at the back-end. This plugin is just a lightweight interface to that back-end. You can examine the guts of the plugin in order to easily figure out how to create similar functionality for your favorite blog platform, wiki, etc. If you do create something, please let us know. We’d love to see what people are building.
We are continuing to experiment with the metadata search function in order to increase its accuracy and flexibility. Again, this might result in seemingly inconsistent behavior. Did we mention that this is a test?
Please note that this API is not meant for bulk harvesting of Crossref metadata. If you need such facilities, then please look at our web site for information about our metadata services.
The data currently behind the plugin is *just* a December 2007 snapshot of our our complete journal article metadata. We have not added books or proceedings yet. We will do so soon and we will start updating the metadata weekly.

We welcome your ideas for tools that we can provide to help researchers. Please, please, please send comments, requests, queries and ideas to us at:

citation-plugin@crossref.org

CLADDIER Final Report

Crossref — Tue, 15 Jan 2008 00:00:00 +0000

I just ran across the final report from the CLADDIER project. CLADDIER comes from the JISC and stands for “CITATION, LOCATION, And DEPOSITION IN DISCIPLINE & INSTITUTIONAL REPOSITORIES”. I suspect JISC has an entire department dedicated to creating impossible acronyms (the JISC Acronym Preparation Executive?)

Anyhoo- the report describes a distributed citation location and updating service based on the linkback mechanism that is widely used in the blogging community.

I think this is an interesting approach and is one that I talked about briefly (PDF) at the UKSG’s Measure for Measure seminar last June. I think that, like most proponents of p2p distributed architectures, they massively underestimate the problem of trust in the network. They fully knowledge the problem of linkback spam, but their hand-wavy-solution(tm) of using whitelists just means the system effectively becomes semi-centralized again (you have to have trusted keepers of the whitelists).

And of course I was mildly exasperated by the report’s characterization of one of the perceived “disadvantages” of the Crossref architectural model being a :

“Centralised service hosting a large persistent store – with the need for a (possibly commercial) business model to justify providing the service.”

Though DOI registries like Bowker and Nielsen Bookdata are commercial, Crossref, the organisation that services the industry that the JISC is concerned with, is *not* a commercial service.

Also if you replaced the phrase “justify providing” with the word “sustain”, the sentence wouldn’t sound like such a “disadvantage.”

But aside from these quibbles, the report makes an interesting (if technical) read.

NLM Blog Citation Guidelines

Geoffrey Bilder — Mon, 15 Oct 2007 00:00:00 +0000

I’ve just returned from Frankfurt Book fair and noticed that there has been some recent in the The NLM Style Guide for Authors, Editors and Publishers recommendations concerning citing blogs.

Which reminds me of an issue that has periodically been raised here at Crossref- should we be doing something to try and provide a service for reliably citing more ephemeral content such as blogs, wikis, etc.?

Personally, I cringe when I see people include plain old URLs (POUs?) in citations. What’s the point? They are almost guaranteed to fail to resolve after a few years. In citing them, you are hardly helping to preserve the scholarly record. You might as well just record the metadata associated with the content.

So why don’t we simply allow individuals to assign DOIs to their content?

As Chuck Koscher says, “Crossref DOIs are only as persistent as Crossref staff.” Crossref depends on its ability to chase down and berate member publishers when they fail to update their DOI records. Its hard enough doing this with publishers, so just imagine what it would be like trying to chase down individuals. In short, it just wouldn’t scale.

But what if we provided a different service for more informal content? Recently we have been in talking with Gunther Eysenbach, the creator of the very cool WebCite service about whether Crossref could/should operate a citation caching service for ephemera.

As I said, I think WebCite is wonderful, but I do see a few problems with it in its current incarnation.

The first is that, the way it works now, it seems to effectively leech usage statistics away from the source of the content. If I have a blog entry that gets cited frequently, I certainly don’t want all the links (and their associated Google-juice) redirected away from my blog. As long as my blog is working, I want traffic coming to my copy of the content, not some cached copy of the content (gee- the same problem publishers face, no?). I would also, ideally, like that traffic to continue to come to to my blog if I move hosting providers, platforms (WordPress, Moveable Type) , blog conglomerates (Gawker, Weblogs, Inc.), etc.

The second issue I have with WebCite is simpler. I don’t really fancy having to actually recreate and run a web-caching infrastructure when there is already a formidable one in existence.

So what if we ran a service for individuals that worked like this:

For a fee, you can assign DOIs to your ephemeral, CC-licensed content.
When you assign a DOI to an item of content (or update an existing DOI), we will immediately archive said content with the Internet Archive (who, incidentally, charges for this service)
We will direct those DOIs to your web site as long as you are both:
Paying the fee
Updating your URLs to point to the correct content
If you fail in either “a” or “b”, we will then redirect said DOIs to the cached version of the content on the Internet Archive (after having warned you repeatedly via automated e-mail).

(Note, as an aside, that we could in theory provide a similar dark-archive service for publishers with non free content using something like JStore as the archive)

This approach would help to ensure that a blogger’s version of content was always linked to as long it was available. It would also preserve the “persistence” of Crossref DOIs by making sure that we could always resolve the DOI even if we were not able to get the owner of said DOI to update it.

So back to the NLM guidelines… On the one hand, I’m delighted to see that the NLM has issued guidelines on citing blogs. It seems glaringly obvious that informal (and ephemeral) content such as blogs and wikis are increasingly becoming vital parts of the scholarly record. On the other hand, it also seems to me that recommending that somebody “cite” with a broken pointer (i.e. a URL) to content verges on tokenism. This isn’t the NLM’s fault- there just isn’t a reliable mechanism for citing informal content in a manner that ensures you can then retrieve and look at said content in the future.

And this is no longer a problem confined to the Scholarly/Professional publishing space. As Jon Udell has occasionally pointed out, citation is increasingly an important currency for *any* professional writer on the web. It seems to me that a system for reliably citing blogs and wikis would benefit many communities. I could easily see commercial hosted Blog services (Blogger, WordPress) offering a “Cached-DOI” feature as a premium service to their clients.

So what do you think? What am I missing? is this something we should be looking at?

Style Guides Recommend DOI strings

Ed Pentz — Wed, 19 Sep 2007 00:00:00 +0000

A couple of recent posts - from A couple of recent posts - from at Jefferson University and IFST at Univ of Delaware- note that the AMA and APA style guides now recommend using a DOI, if one is assigned, in a journal article citation.

A citation in the APA style with a DOI would be:

Conley, D., Pfeiffera, K. M., & Velez, M. (2007). Explaining sibling differences in achievement and behavioral outcomes: The importance of within- and between-family factors. Social Science Research36(3), 1087-1104. doi:10.1016/j.ssresearch.2006.09.002

In the AMA style a reference would be:

Kitajima TS, Kawashima SA, Watanabe Y. The conserved kinetochore protein shugoshin protects centromeric cohesion during meiosis. Nature. 2004;427(6974):510-517. doi:10.1038/nature02312

This is great news. I haven’t looked at the full style guides but it’s not clear if information is given about linking DOIs via http://dx.doi.org.pluma.sjfc.edu/

Information on the APA Style Guide is available - http://apastyle.apa.org.pluma.sjfc.edu/ with specific info on electronic references, URLs and DOIs and here is the AMA info.

This raises the existential question of a DOI as a URI. Is

Conley, D., Pfeiffera, K. M., & Velez, M. (2007). Explaining sibling differences in achievement and behavioral outcomes: The importance of within- and between-family factors. Social Science Research36(3), 1087-1104. doi:10.1016/j.ssresearch.2006.09.002 http://dx.doi.org.pluma.sjfc.edu/10.1016/j.ssresearch.2006.09.002

unnecessary or redundant?