Title is pretty much the description. I’m trying to see if I can download the database and only include data such as name, address, and possibly grid.
Was hoping that this could help me / others who are in areas with no service or no mobile hotspot. Might help with loading folks’ exact address or possibly predicting callsigns?
You can certainly download it as an ADIF, but it might take a long time to load.
We’ve been thinking about doing something like this, but as a separate feature, to preserve the ADIF History for history lookups only.
The main complication is not doing it once, but rather building the mechanisms to update it regularly. And how to make it performant, since downloading and inserting 750k rows on a phone is not ideal.
One thing I have considered, to reduce the size, is to exclude Technician licenses. Yes, sometimes they can activate a summit or a park using VHF, but the benefit of cutting the data in half can outweigh not having a few calls that are actually active.
Another option is to look at pota activators and hunters and only include those, for example.
In any case, if you do go thru with your ADIF experiment, let us know how it went. How big was the file, how long it took to load, etc
You’re telling me you don’t want to sit at a picnic table in the rain hand typing in the FCC database? How dedicated to this are you??
I joke, but I totally understand what you mean from a performance perspective. Doing queries on a database with 750k rows of data would likely be a lot of information.
Your note about only including General / Extra I think is a smart one. I’d argue 10m is probably not the most activated band.
It’d be interesting to trying to gather info on who the most activated Hunter / Activators are, because that would likely clear up 70% of entries you’d want to fill in. I can’t imagine there are a ton of hunters.
I’ve reached out to the owner of hamdb to see if he’s able to provide the information in an adif format. Otherwise, I guess I will try and see if I can download it and format the data myself.
in regards to how often to do it, I’m not sure. People move, people get vanity calls, etc. Once a month? Once a year? Ultimately it’s people’s responsibility to verify the contact info is correct for uploading to POTA I think. Might need some more thinking on this…
I’ll post in here if I get any results.
And way not DX callsign?
Ham world is not restricted to FCC licenses…
This could be a killer feature if well implemented (from performance point of view).
My 2 euro cents
Simo IU3QEZ
Agreed, but I have to start somewhere, right? Definitely not excluding my friends across the pond!
As a side note, I downloaded the ULS from the FCC site for amateurs, and it’s almost 2 million rows of data. Excel got bogged down and not happy lol.
I’ll keep at it and see what I can do.
That’s because ULS is giving you everything in the system. To include expired, cancel, revoked and all of the other non ham licenses that the FCC is responsible for.
I’ll have to work on filtering through the right data and make sure I’m reading it correctly then. Time to do some work.
There is no public database of Italian licenses, for example (and if there is, please share some links!)
As far as I know, only a few countries have public databases. The US, Canada, Germany and the Czech republic. Australia used to, but not anymore.
There are sites like QRZ or HamDB, where individuals have entered their details, but those databases are not public. And even if those companies were willing to share that data with Ham2K, that could be breaking privacy laws in a few jurisdictions, including Europe and California.
This is one area where QRZ has a huge advantage that will be really hard to fight against. Maybe one day, but for now we need to focus on our little logging app.
Agree…
“Effort” vs “Reward”
Stay focused on the App
As an update, I was able to get some code written that takes the entirety of the US database (EN.dat) and puts rows of info with Callsign, Name, Address, City, State, Zip.
It then takes that data and creates a dummy adif file.
It sets the QSO time to January 01, 2001 at 0700 hours on 160M with 59 signal both ways.
Going to try it with a much smaller file size (100 lines or so) and see how the program handles it.
Alrighty, was able to get my python code running.
It looks in the HD.dat file to determine what call signs are active and what are not. It then searches EN.dat for ONLY those active call signs and makes a dummy .adif file.
Total file size right now is 149mb and it takes about 52 seconds to process the database.
Just being the devil’s advocate here for a moment:
If you do this, then you will have worked everyone at least once in the QSO stats. So if you attend to the stats, you will need to adjust the worked N before by minus 1.
If instead, you create a names file from your dataset and import that, you will have the operator name and the stats will be accurate.
And as for my current status, after a year of operating portable with PoLo, I have about 10k real QSOs of real history and anyone new in the stats is actually new to me and I like to know that. I welcome them to the hobby/parks/my log as appropriate. And I feel that’s good to know during the QSO.
If you are after QTH when there is no service, I make do with the Full Lookup option after the operation when I am back in service.
Alan
PS. Also, you do not need RST fields in the history, you can save a lot of file size by ignoring them.
One thing to consider is that you’ll need a grid too.
The ULS database includes address, but not geocoded to latitude and longitude.
PoLo ignores the address (it does display city and state) but uses the “grid” value provided by QRZ.
So you’re going to have to geocode all those addresses, and convert the coordinates to a 6-digit maidenhead grid.
Okay so I have a two part question here for you…
How are historical adif’s handled (looks like there is a function on the app to load it)? Because if all I load is address and name info, that’s fine for my purposes. Does this take a second priority to the most recent info (such as logs I’ve done within ham2k)? What if there is conflicting information between logs?
Second question is related to first. Let’s say I have my info loaded but it’s missing info like grid or whatever. If I get home and do the “lookup all” function, does it overwrite info that was loaded from the historical adif? Will it now default to the data that got looked up?
I’ll work on the code to add grid info I guess 
Historical ADIF is loaded on the database as if they were regular QSOs but with no operation.
When we need to lookup a callsign, the first thing we do is look for the most recent QSO, and use any name, grid and city from that record, if found. This happens whether you’re online or offline.
Then, if online, we query QRZ.com (or any of the other services we support) and update the data with any information we get back.
So, to your questions, data from existing QSOs (and historical ADIF) are looked up first, with the most recent record being used, and then it all gets updated with online lookups if available.
So when you get home and use “lookup all”, it should overwrite any previous results.
