How Do You Keep Track of Your Web-Based Research? 150
time961 asks: "I use the Web extensively to research a wide variety of topics (weird, huh?). However, much of the time I end up printing out web pages and filing them on paper, because that's the easiest way I know to say 'OK, that was interesting, I'll hold on to it until I actually do something about this topic'. Often, I'll run across something that seems relevant to a long-term project or interest and just want to grab it without even reading the details. Paper is OK for reading, browsing, and scribbling, but it's hard to search, it's heavy, and it's wasteful (and I yearn for a day when browsers can reliably print what's on the screen, instead of cutting it off at the margin because some designer doesn't understand layout!). How do others deal with organizing the results of browsing?"
Bookmarks and histories aren't the answer — they're not very good for searching, the UI isn't very good for, say, adding notes, and they don't work offline. Also, stale URLs are a huge problem — a key advantage of paper is that it doesn't randomly fade out in a few days (or decades), so a good solution would have to keep copies, not just references. I imagine something like a FireFox plug-in with a 'Remember This' button and some options for category, keywords, annotations, etc., but I'll bet there are more creative approaches, too."
PDF (Score:5, Informative)
Re:PDF (printer) (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Okay, I just checked, and it allows you to type over the PDF and save it but the free version leaves watermarks.
Re: (Score:3, Informative)
For Windows, I can recommend the following free solutions:
Hope this helps...
Re: (Score:3, Informative)
For Windows there's either the paid route (Adobe Acrobat Suite), or you can use PDFCreator [sourceforge.net] which uses ghostscript. GS used to produce really nasty looking output years ago on Windows (circa the late 90's), but that's not the case anymore.
For linux, print to ps then use something like ps2pdf (once again GhostScript).
Re: (Score:3, Informative)
That's it, the printer stores the PDF files by default to ~/PDF but you can change this location in
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
But, in general, creating pictures of html pages (whether on actual paper or as postscript files) seems like a really bad idea. HTML files are small, searchable, portable, and easily transformed into other formats. Postscript files are none of these things.
It may be less wasteful than generating paper (given some assumption about the environmental and economic costs associated with hard drive space), but an html based solution seems l
Re: (Score:2)
Actually, PostScript and PDF are all of those things. Unfortunately, some applications will create a static image embedded in a PS/PDF, but they can certainly contain formatted text that can be searched, indexed, compressed, and transformed. In practice, PostScript and PDF have a lot in common with HTML. Although the languages are vastly different, they're functionally similar
Re: (Score:2)
Media Server (Score:4, Funny)
Re: (Score:2)
I have a friend who is an attorney and who performs extensive research against a wide array of source material...including the web...and he swears by Microsoft OneNote:
http://office.microsoft.com/en-us/onenote/default. aspx [microsoft.com]
Yes, a Microsoft product...let the flames begin.
Errrr (Score:1)
Of course one problem with them is that they can disappear or change between the moment you save them and the moment you use them. The obvious answer is to save a local copy (with wget, or whatever..) which will be easier to search than a paper... And you can still print it if you need.
Then you can easily search though all the pages you downloaded for the one which holds the information you need, which probably takes you a long time with paper...
Of course all those things are b
Re: (Score:2)
Bookmarks and histories aren't the answer
You:
Ever heard of bookmarks
You lose at slashdot!
Re: (Score:2)
Lose? I think not.. He's taken the slashdot evolution to the next level and doesn't even bother to read the summary..
New: Google Notebook (Score:5, Informative)
Re: (Score:2)
That said, it does look pretty interesting, so I'm gonna give it a whirl..
Re: (Score:2, Informative)
I was surprised not to see Google Notebook as one of the first answers, as it indeed works very well for organizing material found on the web. I guess, Slashdot is less of a Google fan club than many people assume it to be!
The FF extension makes saving "permanent" pages easy via a right-click option. For pages that may become inaccessible over time, the content of interest can be copy-pasted directly into the Notebook entry. And Google search options coupled with the possibility of creating multiple Notebo
Re: (Score:2)
Re: (Score:2)
Given that the very first response has the title
(OT, but why did the guy who created Slashdot's CSS format inline <quote> as a block element? That's what <blockquote> is for.)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2, Funny)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Does Beagle support Mozilla Thunderbird?
Beagle is no longer built with default Thunderbird support, as of version 0.2.15. (Support for indexing email, news, RSS, and addresses had originally been added in Beagle 0.2.8, but was removed due to memory issues.)
I'm currently looking at setting up a homebrew system using htdig. I run my own server, so in theory I should be able to run htdig on it, and run a c
Re:Errrr (Score:4, Informative)
Ask and ye shall receive!
http://bookmarkdd.mozdev.org/ [mozdev.org]
Or the Mozilla Addons page for it :
https://addons.mozilla.org/en-US/firefox/addon/15
Re: (Score:2)
the badness of bookmarks. (Score:2)
Worse yet, sometimes I'll bookmark a page and go back to it, and the page will be gone, the site down, or changed to something which isn't useful to me.
Hey, it's the 21st century (Score:1)
Basket (Score:1)
Re: (Score:2)
Zotero (Score:4, Interesting)
Re: (Score:2)
Re: (Score:2, Informative)
Bookmarks (Score:2)
Seriously? (Score:3, Informative)
You can either keep what you save in some sort of logical arrangement, or trust your handy desktop search engine to find it for you later (though that seems to reduce the problem back to finding the info in the first place, though at least you don't need to worry about the content going offline at some future date.
Re: (Score:2)
I either e-mail it to myself (or my working group) or I'll blog about it. It's a very low-down and dirty system, but the search tools in my mail clients are good enough to let me find things as long as I know the year (I archive by year).
Although the Scrapbook extension for Firefox sounds very intriguing.
File....System! (Score:2)
That's the best idea. The filesystem is the most robust database I've found and hierarchies work well for me.
This story would be saved in 'computers > networking > internet > sites > slashdot.org > stories > ask > obvious', for instance.
Anything I come up with goes on the blog. In theory, then somebody could do the same (were there people who cared to read what I write).
Easy (Score:5, Funny)
What's wrong with Bookmarks? (Score:1, Flamebait)
Use folders and subfolders to organise them, and if you really honestly have an unmanageably large number of bookmarks that you couldn't possibly just google again later, cut and paste from the bookmarks file to any kind of saveable text document.
Re: (Score:2)
Google Desktop, PDF, directory organization (Score:2)
Google Desktop Search and PDF. GDS does the indexing, PDF preserves the original page.
A good use of directories for organizing helps to avoid "lost" files from floating around. I use this for research papers and projects.
Re: (Score:2)
The last time I tried Google Desktop Search, I found it to be useless if you had more then a few hundred items to be indexed. (Think in the range of tens of thousands of files.)
Copy paste (Score:1)
PDF! (Score:4, Insightful)
Re: (Score:2)
http://web.archive.org/web/20011217172330/http://
Re: (Score:2)
Re: (Score:2)
Mindmapping (Score:1)
Have you tried Acrobat? (Score:1)
Re: (Score:2)
Research trails (Score:2)
PDF/Annotating (Score:2)
For the annotations I would suggest the FoxIT [foxitsoftware.com] PDF reader (free) and buy the Pro Pack [foxitsoftware.com] [US$40 ](one of the few softwares I have found so useful and at good price to actually buy) which will allow you to add annotations and mark the text among other things.
I will use this post to ask if anyone knows of an open source alternative to this the P
Save Page as ... (Score:1)
Then just get Google Desktop or something similar to index those.
Re: (Score:2)
Recommend good free PDF printer? (Score:1)
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:2)
Take a look at the ScrapBook Firefox extension (Score:5, Informative)
You can find all the features in a nice list [vis.ne.jp] at the official homepage with tons of pretty screenshots. There's even a 50 page manual [vis.ne.jp] (PDF) created by Andrew Giles-Peters.
Even though development has seemingly halted since December 2005, it's still one of the most well rounded extensions for Firefox I've come across yet.
Re: (Score:2, Interesting)
I second this. I MUCH prefer Scrapbook to PDF saves, which I used earlier, because Scrapbook preserves all the original HTML and the format of images (whereas PDF converts them and makes them hard to separate out), is also searchable/indexable by whatever indexing program you want, and can be highlighted, annotated, etc.
Let's just hope they keep developing, at least enough to ensure that it continues to work with future releases of Firefox. My sense is that they are, given that the developers blog at htt [xuldev.org]
Re: (Score:2)
You're only a year off
Jonah HEX
Re: (Score:2)
Re: (Score:3, Insightful)
In years past, I used PDFs, but since 2003, I have been using scrapbook.
Personally, I use it for vacations and business trips. When I'm on on the road, I just 'scrapbook' important pages (like Google map directions) and when I need to pull something up, I just open the laptop. Now, on the other hand, its a lot easier to pull the PDF files over to your PDA...
Now-a-days, I use this less frequently due to the rise of high speed cellular internet, but its still extremely useful for
Word Processor (Score:1)
Scrapbook (Score:1)
Helps you to save Web pages and organize the collection.
I wget it! (Score:4, Informative)
At the moment, I have on order of 10GB just of websites, radio clips, and what have you that I have used for previous research. Not only that but I can also maintain a simple directory structure and never have to worry that that "firefox plugin" will still be compatible with version 4.765.
Another neat function is you can specify just a particular files (www.whatever.com/pic.jpg), or all the files with a particular extension *.jpg, or only the files in that directory. You can also use it to spider (limited) all the links on a site. Though be kind and don't do this too often, as I am sure it eats a lot of bandwidth.
The last (and greatest) thing, is it remains in a well-known and easily editable format.
Alternatively, I have also used a MediaWiki setup so that I could drop down notes for classes, or other interesting things in it, but this required substantially more overhead than wget.
Re: (Score:2)
In any case, it's not a flawless method because there are many sites that can't be downloaded in whole, due to them using Javascript links or dynamic content that confu
Re: (Score:2)
Since you're already viewing the site in a browser, why would you LEAVE the browser to go to a CLI to do something the browser already has b
Re: (Score:2, Insightful)
you pretty much only get what is in the 'public' directory
stylesheets, javascript, images...
A personal wiki? (Score:3, Interesting)
I also save web pages as "Web Page, Complete". It now occurs to me that I should make a specific directory for those pages.
MediaWiki, plus my own pico-Google (Score:2)
I also wrote a sort of pico-Google in PHP/MySQL a couple years back, and I still use that regularly. It's a sort of searchable bookmark database. I feed it a URL, it goes out to the page and sucks down all the text, normalizes it, and breaks it into keyw
Re: (Score:2)
DEVONthink (Score:2, Informative)
Using a good PDF exporter (I'm on OS X, so look elsewhere for free & easy ways to do this on Windows), DEVONthink will pretty much keep everything organized like a digital filing cabinet.
'Course, the cheapest version costs $39.95, but I can attest to the fact that this software WORKS (I got it heavily discounted in the MacHeist 2006 bundle).
Re: (Score:2)
Free & easy ways to do this on Windows:
Hope this helps...
Yojimbo (Score:2, Interesting)
You can either save a "web archive", which is the web page incl. all graphics/css/etc., or a PDF of the page (nicely integrated into print services). Both document types are rendered inside the app and are searchable. Yojimbo has also tags and folders to keep things organised. And you can also save regular notes (formated and with images). Covers all bases.
When it comes to pure PDF, YEP http://www.yepthat.com/ [yepthat.com] is an excellent alternative. Kind o
Re: (Score:2)
I second the Yojimbo recommendation. A few more points: A third item type for dealing with web material in Yojimbo is regular bookmarks, which can have the same tags and/or labels as every other Yojimbo item. Yojimbo uses an SQLite database to store all its data, so you can't use the Finder to get at Yojimbo's PDF's or Web Archives.
JP
Webforia Organizer .... (Score:2)
Apparently Webforia went out of business some time ago and the software no longer works.. I believe it had limited functionality with IE 6, but not enough to make it
Bookmarks plus a whiteboard (Score:2)
If I get any "aha" ideas while reading these papers, I record them in a whiteboard or notebook. Eventually, I have the paper distilled to three or four of these
Evernote (Score:3, Informative)
It's a program that allows you to easily save a copy of just about anything (certainly anything on the web...) with links to the original and everything else. The notes are automatically stored in chronological order for browsing. You can also apply tags to your liking and it has full search capabilities as well. It's free for the regular version, if you want to import handwritten notes and have them be searchable as well there's a charge.
It's awesome and I think fits your needs exactly, or at least I use it to meet the needs you described and I've had no problems with it.
Now if I could just force myself to go back and do something with the research later...
P.S. There's a writer in The Atlantic named James Fallows who has a column on useful technology tools. That's where I first learned of Evernote. He had several other suggestions to fit the bill in that column and more generally, he's usually worth a read.
Firefox Bookmarks (Score:3, Interesting)
In my bookmarks folder I have a "Projects" folder.
Within my "Projects" folder I have an alphabetic listing of folders with each project's name.
If the project is small, I fill it directly with book marks. I do take the time to add notes, because if the URL does go stale, the notes will let me know what I'm now missing. More often than not, missing information can be replaced in the future with another URL that has the same or more up-to-date information. Additionally Google Desktop searches my bookmarks file, so I just double-click ctrl and can search via keywords that way.
This whole setup is a bit of a hack, but it's worked. I'm hoping either Firefox 3.0 will have a fantastic bookmark manager or a plug-in author creates something truly wonderful for the existing bookmark system.
Opera Notes (Score:2, Insightful)
Comment removed (Score:3, Informative)
Re: (Score:2)
Re: (Score:2)
Text (Score:2)
I do this as regularly as anyone.
lynx -dump > ~/docs/filename
or if you're organised
lynx -dump | add_to_database_script
What's important to me is the content itself, not the "web content", so an attorney, for example, would take a very different approach (typically a hard copy that can be filed, duplicated, etc.). Note that unless you work for a law firm or a well-run business, managing pap
Personal wiki (Score:2)
Why on earth would you want to print a webpage? (Score:2)
- full-text search, on one or multiple files
- text and other elements can be copied off the page
- links still work
Pity Windows doesn't attach comments to a file: in Mac OS 9 at least, if you saved a Web page, the page URL would end up as a comment (viewable by doing Get Info on the file).
Zoot (Score:3, Informative)
Cheap dig. (Score:2)
This is so unfair. Are you a webdesigner? Are you even a designer at all? If you've ever done both print design and web design, you will appreciate how much more challenging web design is. Imagine designing for a completely unique for every viewer canvas, rather than, say, 10,000 identical copies of a newspaper or print ad. You have to allow
OneNote 2007 (Score:3, Informative)
It comes with a "print to..." driver so you can print to your OneNote notebook, and provides a good framework for organizing your notes, and you don't need to kill as many trees as printing to paper.
Another possibility is to get a PDF printer; you can either just organize your notes with file system folders, or if you want something a little bit more useful to track relations between different items, you can use something like PersonalBrain [thebrain.com] to for organization.
What I use... (Score:2)
It's basically a delicious clone but the feature I love the most is the snapshotting one. That way I never have to worry about the information going missing. It's been very useful for things that are hosted on university servers that disappear when the student leaves. Some of my bookmarks are private while others are public. They provide a javascript snippet you can put in your toolbar to bookmark the current page.
It requires a server of your own to host
Copy URL + helps a lot (Score:3, Informative)
Copy URL +
"The Copy URL+ extension enables you to copy to the clipboard the current
document's address along with additional information such as the document's
title, the current selection or both."
https://addons.mozilla.org/en-US/firefox/addon/12
It installs a context-menu, allowing you to copy any or all of page title, URL, and most importantly: the text currently selected.
At other times, I use bookmarks in a new folder specific to the subject. You can add keywords to bookmarks in FF.
Just googling around, or doing proper research? (Score:2)
It's not worth archiving everything. But what you do archive, you should archive properly and carefully. All interesting information falls into two groups: