Оглавление
Due to the various protections (Recaptcha, dynamic content, IP checks, etc.) scraping of Google Maps in pure python is a challenging task even if you have experience in the field. Fortunately, there are many tools that you can employ to scraper the reviews in python or any other programming language.
In this article, you will see the two most common tools of scraping Google Maps: browser emulation and Outscraper API. This tutorial should be sufficient to extract data for individuals as for big data teams.
Scrape Google Places in Python
You can try scraping Google with pure python but within 10 requests you will see that your software is blocked. This happens since Google validates sessions and expects JavaScript to make some API calls in the background. Therefore, using browser emulation is required to prevent Google Maps from blocking our crawler. Additionally you can use proxies.
To get started with building the google maps scraper with browser emulation please follow the steps described in this tutorial How to Scrape Google Maps With Python and Selenium.
Scrape Google Places in Python Within 6 Minutes
Developing Google Maps scraper and maintaining it might require a great amount of time and effort. Therefore, many engineering teams find it beneficial for them to focus on their main field and to use Outscraper Platfom as a reliable source of data for their apps.
Scrape Google Maps By Using SDK
1. Вам понадобятся python3 + и этот пакет Python. Install the package by running the following command.
pip install google-services-api
2. Получите свой ключ API от страница профиля.
3. Import the package and initialize it with the key. Specify queries to run the search like this "Рестораны поблизости Лос-Анджелес, США". Use countries and cities inside each query.
from outscraper import ApiClient
api_cliet = ApiClient(api_key='KEY_FROM_OUTSCRAPER')
response = api_cliet.google_maps_search(
'Restaurants near Los Angeles, USA',
language='en',
region='es',
limit=100
)
4. Wait a few seconds till the data will be fetched.
[
{
"name": "Sushi Gen",
"full_address": "422 E 2nd St, Los Angeles, CA 90012",
"borough": "Little Tokyo",
"street": "422 E 2nd St",
"city": "Los Angeles",
"postal_code": "90012",
"country_code": "US",
"country": "United States of America",
"us_state": "California",
"state": "California",
"plus_code": null,
"latitude": 34.0466917,
"longitude": -118.2385444,
"time_zone": "America/Los_Angeles",
"site": "http://www.sushigen-dtla.com/",
"phone": "+1 213-617-0552",
"type": "Sushi restaurant",
"subtypes": "Sushi restaurant, Japanese restaurant",
"posts": null,
"rating": 4.6,
"reviews": 1244,
"reviews_data": null,
"photos_count": 1261,
"google_id": "0x80c2c63850150b9d:0x96cdcf9a38a24170",
"place_id": "ChIJnQsVUDjGwoARcEGiOJrPzZY",
"reviews_link": "https://search.google.com/local/reviews?placeid=ChIJnQsVUDjGwoARcEGiOJrPzZY&q=Restaurants+near+Los+Angeles,+USA&authuser=0&hl=en&gl=US",
"reviews_id": "-7580174336512605840",
"photo": "https://lh5.googleusercontent.com/p/AF1QipMtt0bMtJTF1bDTZ2tbXuWXtZTNWmvjJUu3ejKy",
"working_hours_old_format": "Monday: Closed | Tuesday: 11AM\u20132PM,5\u20137:30PM | Wednesday: 11AM\u20132PM,5\u20137:30PM | Thursday: 11AM\u20132PM,5\u20137:30PM | Friday: 11AM\u20132PM,5\u20138PM | Saturday: 4\u20138PM | Sunday: Closed",
"working_hours": {
"Monday": "Closed",
"Tuesday": "11AM\u20132PM,5\u20137:30PM",
"Wednesday": "11AM\u20132PM,5\u20137:30PM",
"Thursday": "11AM\u20132PM,5\u20137:30PM",
"Friday": "11AM\u20132PM,5\u20138PM",
"Saturday": "4\u20138PM",
"Sunday": "Closed"
},
"business_status": "OPERATIONAL",
"about": {
"Service options": {
"Takeout": true,
"Delivery": false
},
"Health & safety": {
"Mask required": true
},
"Highlights": {
"Great tea selection": true
},
"Popular for": {
"Lunch": true,
"Dinner": true,
"Solo dining": true
},
"Accessibility": {
"Wheelchair accessible entrance": true
},
"Offerings": {
"Healthy options": true,
"Small plates": true
},
"Dining options": {
"Dessert": true
},
"Amenities": {
"High chairs": true
},
"Atmosphere": {
"Casual": true,
"Cozy": true
},
"Crowd": {
"Groups": true,
"Tourists": true
},
"Planning": {
"Dinner reservations recommended": true,
"Accepts reservations": true,
"Usually a wait": true
}
},
"range": "$$$",
"reviews_per_score": {
"1": 25,
"2": 14,
"3": 52,
"4": 291,
"5": 862
},
"reserving_table_link": null,
"booking_appointment_link": null,
"owner_id": "100930306416993024046",
"verified": true,
"owner_title": "Sushi Gen",
"owner_link": "https://www.google.com/maps/contrib/100930306416993024046",
"location_link": "https://www.google.com/maps/place/Sushi+Gen/@34.0466917,-118.2385444,14z/data=!4m8!1m2!2m1!1sSushi+Gen!3m4!1s0x80c2c63850150b9d:0x96cdcf9a38a24170!8m2!3d34.0466917!4d-118.2385444"
},
{
"name": "Water Grill",
"full_address": "544 S Grand Ave, Los Angeles, CA 90071",
"borough": "Financial District",
"street": "544 S Grand Ave",
"city": "Los Angeles",
"postal_code": "90071",
"country_code": "US",
"country": "United States of America",
"us_state": "California",
"state": "California",
"plus_code": null,
"latitude": 34.0490403,
"longitude": -118.25470659999999,
"time_zone": "America/Los_Angeles",
"site": "https://www.watergrill.com/la",
"phone": "+1 213-891-0900",
"type": "Seafood restaurant",
"subtypes": "Seafood restaurant",
"posts": null,
"rating": 4.4,
"reviews": 1387,
"reviews_data": null,
"photos_count": 1429,
"google_id": "0x80c2c7b4901ce7c7:0x2ad996c9a82b097c",
"place_id": "ChIJx-cckLTHwoARfAkrqMmW2So",
"reviews_link": "https://search.google.com/local/reviews?placeid=ChIJx-cckLTHwoARfAkrqMmW2So&q=Restaurants+near+Los+Angeles,+USA&authuser=0&hl=en&gl=US",
"reviews_id": "3087664812393171324",
"photo": "https://lh5.googleusercontent.com/p/AF1QipPX7-DP07mvQQSxDjDjuOVYj9HlQ2bvNJ_EEZYW",
"working_hours_old_format": "Monday: 11:30AM\u20139PM | Tuesday: 11:30AM\u20139PM | Wednesday: 11:30AM\u20139PM | Thursday: 11:30AM\u20139PM | Friday: 11:30AM\u201310PM | Saturday: 4\u201310PM | Sunday: 4\u20139PM",
"working_hours": {
"Monday": "11:30AM\u20139PM",
"Tuesday": "11:30AM\u20139PM",
"Wednesday": "11:30AM\u20139PM",
"Thursday": "11:30AM\u20139PM",
"Friday": "11:30AM\u201310PM",
"Saturday": "4\u201310PM",
"Sunday": "4\u20139PM"
},
"business_status": "OPERATIONAL",
"about": {
"Service options": {
"Takeout": true,
"Dine-in": true,
"Delivery": false
},
"Health & safety": {
"Mask required": true,
"Staff wear masks": true,
"Staff get temperature checks": true,
"Staff required to disinfect surfaces between visits": true
},
"Highlights": {
"Great dessert": true
},
"Popular for": {
"Lunch": true,
"Dinner": true,
"Solo dining": true
},
"Accessibility": {
"Wheelchair accessible entrance": true,
"Wheelchair accessible seating": true
},
"Offerings": {
"Alcohol": true,
"Beer": true,
"Cocktails": true,
"Coffee": true,
"Hard liquor": true,
"Healthy options": true,
"Late-night food": true,
"Small plates": true,
"Vegetarian options": true,
"Wine": true
},
"Dining options": {
"Lunch": true,
"Dinner": true,
"Dessert": true,
"Seating": true
},
"Amenities": {
"Bar onsite": true,
"High chairs": true,
"Restroom": true,
"Wi-Fi": true
},
"Atmosphere": {
"Cozy": true,
"Romantic": true,
"Upscale": true
},
"Crowd": {
"Family-friendly": true,
"Groups": true,
"Tourists": true
},
"Planning": {
"Lunch reservations recommended": true,
"Dinner reservations recommended": true,
"Accepts reservations": true
}
},
"range": "$$$",
"reviews_per_score": {
"1": 27,
"2": 32,
"3": 97,
"4": 408,
"5": 823
},
"reserving_table_link": "https://www.watergrill.com/la#RESERVATIONS",
"booking_appointment_link": "https://www.watergrill.com/la#RESERVATIONS",
"owner_id": "116715188720123884576",
"verified": true,
"owner_title": "Water Grill",
"owner_link": "https://www.google.com/maps/contrib/116715188720123884576",
"location_link": "https://www.google.com/maps/place/Water+Grill/@34.0490403,-118.25470659999999,14z/data=!4m8!1m2!2m1!1sWater+Grill!3m4!1s0x80c2c7b4901ce7c7:0x2ad996c9a82b097c!8m2!3d34.0490403!4d-118.25470659999999"
},
...
]
Video Tutorial on How to Scrape Google Maps in Python
Bonus
Sign up now and use the promotion code OUTSCRAPER1 to get the - 10% скидка.Пакет Python ► https://pypi.org/project/google-services-api Google Maps Places API ► https://outscraper.com/google-maps-api
0 Комментарии